Deciphering Q*: Unveiling the Next Evolution in AI's Self-Learning Journey

November, 2023

In the span of a decade immersed in the realm of artificial intelligence, a particular algorithm has captured the collective imagination like none before. It's a name that conjures visions of the future, yet it exists without a published paper, without statistics, without a tangible product. It's a phantom of potential, a specter of what could be. But let us not be seduced by phantasms. Instead, let's deconstruct this enigma, piece by piece, and in doing so, perhaps we can grasp the essence of this AI dream.

The dance of Search and Learning in AI is not new. It's a tango that began in earnest with AlphaGo's historic victory in 2016. That machine, which humbled a human world champion, was not just a program—it was a harbinger of the future. To understand it, we must dissect its anatomy, which comprised four key components: a Policy Neural Network and a Value Neural Network, both dedicated to learning; the Monte Carlo Tree Search, a methodical, ponderous search; and a groundtruth signal, the simple binary of win or loss, fueling the system's relentless drive to improve.

AlphaGo was a self-improving entity. It played itself, learned from itself, and in doing so, it became more than the sum of its parts. It was a perpetual motion machine of artificial intelligence, a closed loop of self-enhancement that did not merely mimic human play but transcended it.

Now, let us turn our gaze to the enigmatic Q*. If we were to reverse-engineer this fantasy, we might imagine it as AlphaGo's spiritual successor, with its own quartet of components. Its Policy Neural Network would be a powerful internal generative model, crafting solutions to complex problems. The Value Neural Network would judge the likelihood of each step in a reasoning chain, a critical evaluator in the process of thought.

In May 2023, a paper titled "Let's Verify Step by Step" hinted at the evolution of these ideas. It introduced Process-supervised Reward Models, a method of providing feedback for each step in a reasoning sequence, as opposed to judging an entire output at once. This granularity allows for more precise shaping of the machine's behavior, a more nuanced approach to teaching an AI to think.

The search component for Q* would need to navigate the vast, intricate landscape of "all reasonable strings," a far cry from the discrete game states of Go. The research community has already begun to explore this with concepts like the Tree of Thought and the Graph of Thought, nonlinear extensions of the Chain of Thought that promise even more sophisticated search capabilities.

And what of the groundtruth signal for Q*? It could be as straightforward as the known answers to math problems, or perhaps a more complex system like a formal verification tool that turns mathematics into a coding problem, providing compiler-like feedback.

As with AlphaGo, the iterative improvement of the Policy and Value Neural Networks, informed by human expertise, could create a self-improving loop. A better policy aids the search, which in turn gathers better data for the next round of learning.

It's important to note that this is all about reasoning. Creativity, the domain of poetry and humor, is a fundamentally human endeavor. I believe that natural data, the product of human minds, will continue to outshine synthetic creations.

The pursuit of artificial general intelligence is a marathon, not a sprint. DeepMind's Gemini project, which seeks to use "AlphaGo-style algorithms" to enhance reasoning, is a testament to the industry's direction. Google, too, will undoubtedly keep pace, if not lead the charge.

In this journey, we must remember that the essence of creativity, the spark that ignites innovation, is uniquely human. As we advance, we must guard against over-reliance on artificial intelligence, lest we lose the very thing that makes us who we are.


In the end, the Q* fantasy is just that—a fantasy. But within it lies the kernel of truth about our future with AI: a future where machines learn not just from us, but from themselves, growing in ways we can barely imagine. And as we stand on the precipice of this new era, we must be both the creators and the custodians of the intelligence we bring into this world.

The Immense Potential of Fine-Tuning AI: Highlights from the 'Anything But Wrappers' Hackathon

August, 2023

The "Anything But Wrappers" hackathon recently took place in San Francisco, where the city's top AI engineers gathered to explore the capabilities of fine-tuning. The results were nothing short of impressive, showcasing the immense potential of AI in various fields.

Standout Projects


The "Anything But Wrappers" hackathon was a testament to the immense potential and versatility of AI. The event showcased how fine-tuning can be used to create practical solutions, automate tasks, and even add a dash of creativity to AI applications. As AI continues to evolve, we can only expect more groundbreaking developments in the future.

The Business-Technology Gradient

July 22, 2023

The Business-Technology Gradient

From Business Strategy to Technology Implementation

It's not always easy to understand the subtle nuances that exist between the business end and the technology end of projects. Where does one end and the other begin? How do they intertwine and interact? And most importantly, how can we navigate this spectrum to deliver the most effective solutions to our clients?

Welcome to the concept of the Business-Technology Gradient. This is a way to visualize and understand the multifaceted nature of web development projects. We break down the whole process into four categories: Business, Management, Design, and Technology. By understanding these categories and how they interact, we can better plan, manage, and execute our projects, creating solutions that are not only technically sound but also closely aligned with business goals. Let's dive in.

  1. Business

    This category forms one end of the spectrum and represents high-level, strategic planning and decision-making activities. This is the stage where broad business objectives are determined, and overarching market strategies are developed. Key activities include defining the product-market fit, establishing unique selling propositions (USPs), and setting out marketing strategies. Sales planning and the analysis of return on investment (ROI) also fall under this umbrella. It's all about understanding the market, identifying opportunities, and positioning the business to leverage these opportunities. For example, in a software development context, this might involve identifying a market need for a particular kind of software solution, determining how your solution will be unique or better than existing options (the USP), and laying out strategies for reaching and convincing potential customers.

  2. Management

    As the bridge between business and technology, this category involves coordinating and managing how the business's strategic goals are translated into technological solutions. This includes client and project management, where you'd coordinate with clients to understand their needs and manage the resources, timelines, and activities needed to deliver a solution. It also includes risk management, where you'd identify potential challenges or roadblocks in a project and plan how to avoid or mitigate them. This is about ensuring that business strategies are effectively carried out through technology.

  3. Design

    This category is split into two distinct but related subcategories:

    • Business Design

      This involves the initial stages of designing a tech solution based on business needs. Key tasks include business analysis, where you'd examine the business's operations and identify how a tech solution could improve them; requirements gathering, where you'd define what the solution needs to do to provide value; and creating functional specifications, where you'd outline the specific features and functions the solution will have. This is about translating the high-level business strategy into a detailed plan for a technology solution.

    • Technical Design

      This is where the tech solution starts to take shape. You'd be making key decisions about the software architecture, or how the solution will be structured and organized; designing the user interface to ensure a good user experience; and conducting entity-relationship modeling, or outlining how different elements of the solution will interact with each other. This stage sets the groundwork for the actual coding and building of the tech solution.

  4. Technology

    This is the other end of the spectrum, where the actual creation, implementation, and maintenance of the tech solution happen. This involves web development and programming, using specific tools and languages (like Ruby, JavaScript, SQL, etc.) to build the software. It also includes working with databases and servers (like PostgreSQL, Nginx, etc.), as well as DevOps activities to streamline the development process and ensure the solution is reliable and efficient. This is all about turning the design and plans from the earlier stages into a working, effective technology solution.

Activities and the Business-Technology Gradient

Here are examples of the types of activities that fall within each category of the business-technology gradient:

  1. Business:
    • Market Research
    • Business Model Design
    • Strategic Planning
    • Product Development Planning
    • Marketing Strategy Formulation
    • Sales Strategy and Planning
    • Financial Forecasting
    • Competitive Analysis
    • Pricing Strategy
    • Key Performance Indicators (KPIs) Definition
  2. Management:
    • Project Management
    • Stakeholder Management
    • Resource Allocation
    • Timeline and Schedule Management
    • Budget Management
    • Risk Assessment and Management
    • Contract Negotiation and Management
    • Communication and Coordination
    • Quality Assurance Planning
    • Project Documentation
  3. Design:
    • Business Design:
      • Business Analysis
      • Requirement Gathering
      • Use Case Development
      • Functional Specification Documentation
      • Workflow Design
      • User Story Creation
      • Wireframing
      • Prototyping
      • Business Process Re-engineering
      • Data Analysis and Reporting
    • Technical Design:
      • System Architecture Design
      • User Interface (UI) / User Experience (UX) Design
      • Entity-Relationship Modelling
      • Database Design
      • Network Design
      • Security Design
      • Performance Modeling
      • Technology Selection
      • Integration Planning
      • Test Planning
  4. Technology:
    • Programming (Python, Java, etc.)
    • Web Development (HTML, CSS, JavaScript, etc.)
    • Mobile App Development (Swift, Kotlin, etc.)
    • Database Management (SQL, PostgreSQL, etc.)
    • Cloud Management (AWS, Azure, Google Cloud, etc.)
    • Server Management (Nginx, Apache, etc.)
    • Security Implementation
    • Software Testing
    • Deployment and Continuous Integration
    • Maintenance and Support

This list is a guide; specific tasks can differ based on unique project needs and business requirements.

In a nutshell, these categories cover the full range from strategic planning to detailed technical implementation, with each category building on the previous ones to deliver a cohesive, effective tech solution.