November, 2023
In the span of a decade immersed in the realm of artificial intelligence, a particular algorithm has captured the collective imagination like none before. It's a name that conjures visions of the future, yet it exists without a published paper, without statistics, without a tangible product. It's a phantom of potential, a specter of what could be. But let us not be seduced by phantasms. Instead, let's deconstruct this enigma, piece by piece, and in doing so, perhaps we can grasp the essence of this AI dream.
The dance of Search and Learning in AI is not new. It's a tango that began in earnest with AlphaGo's historic victory in 2016. That machine, which humbled a human world champion, was not just a program—it was a harbinger of the future. To understand it, we must dissect its anatomy, which comprised four key components: a Policy Neural Network and a Value Neural Network, both dedicated to learning; the Monte Carlo Tree Search, a methodical, ponderous search; and a groundtruth signal, the simple binary of win or loss, fueling the system's relentless drive to improve.
AlphaGo was a self-improving entity. It played itself, learned from itself, and in doing so, it became more than the sum of its parts. It was a perpetual motion machine of artificial intelligence, a closed loop of self-enhancement that did not merely mimic human play but transcended it.
Now, let us turn our gaze to the enigmatic Q*. If we were to reverse-engineer this fantasy, we might imagine it as AlphaGo's spiritual successor, with its own quartet of components. Its Policy Neural Network would be a powerful internal generative model, crafting solutions to complex problems. The Value Neural Network would judge the likelihood of each step in a reasoning chain, a critical evaluator in the process of thought.
In May 2023, a paper titled "Let's Verify Step by Step" hinted at the evolution of these ideas. It introduced Process-supervised Reward Models, a method of providing feedback for each step in a reasoning sequence, as opposed to judging an entire output at once. This granularity allows for more precise shaping of the machine's behavior, a more nuanced approach to teaching an AI to think.
The search component for Q* would need to navigate the vast, intricate landscape of "all reasonable strings," a far cry from the discrete game states of Go. The research community has already begun to explore this with concepts like the Tree of Thought and the Graph of Thought, nonlinear extensions of the Chain of Thought that promise even more sophisticated search capabilities.
And what of the groundtruth signal for Q*? It could be as straightforward as the known answers to math problems, or perhaps a more complex system like a formal verification tool that turns mathematics into a coding problem, providing compiler-like feedback.
As with AlphaGo, the iterative improvement of the Policy and Value Neural Networks, informed by human expertise, could create a self-improving loop. A better policy aids the search, which in turn gathers better data for the next round of learning.
It's important to note that this is all about reasoning. Creativity, the domain of poetry and humor, is a fundamentally human endeavor. I believe that natural data, the product of human minds, will continue to outshine synthetic creations.
The pursuit of artificial general intelligence is a marathon, not a sprint. DeepMind's Gemini project, which seeks to use "AlphaGo-style algorithms" to enhance reasoning, is a testament to the industry's direction. Google, too, will undoubtedly keep pace, if not lead the charge.
In this journey, we must remember that the essence of creativity, the spark that ignites innovation, is uniquely human. As we advance, we must guard against over-reliance on artificial intelligence, lest we lose the very thing that makes us who we are.
In the end, the Q* fantasy is just that—a fantasy. But within it lies the kernel of truth about our future with AI: a future where machines learn not just from us, but from themselves, growing in ways we can barely imagine. And as we stand on the precipice of this new era, we must be both the creators and the custodians of the intelligence we bring into this world.
August, 2023
The "Anything But Wrappers" hackathon recently took place in San Francisco, where the city's top AI engineers gathered to explore the capabilities of fine-tuning. The results were nothing short of impressive, showcasing the immense potential of AI in various fields.
One of the standout projects was by an engineer named Harrison, who fine-tuned GPT-3.5 on his emails. The goal was to create an AI that could politely reject notes from venture capitalists and respond to other industry professionals. The result was a model that could handle email communication with finesse and tact, a tool that could potentially save a lot of time for busy professionals.
Another intriguing project involved launching a multitude of cloud resources simultaneously to rack up a considerable bill. This idea may not be practical for everyday use, but it demonstrates the power and scalability of AI. The project also gave a nod to Alphachive, a tool used by Stanford researchers to rate academic papers.
The hackathon also saw the birth of a unique crypto salesman AI. This model was fine-tuned to answer questions while simultaneously promoting cryptocurrency. While this may raise ethical questions about the use of AI in sales and marketing, it undeniably showcases the versatility of AI.
The event also highlighted the power of distributed training. One experiment involved training a LoRA on 128 GPUs on different machines with only 50% overhead. This project, led by Eric Yu, demonstrates the potential of AI in handling complex computations and large data sets.
One of the most practical applications came from a fine-tuned model that learned to lint JavaScript better than GPT-3.5 and even GPT-4. This Llama Linter could potentially revolutionize coding by automating the process of debugging and improving code quality.
Adding a dash of creativity to the mix, a fine-tuned GPI-3 was used to generate ASCII art from any text prompt. This project, while not necessarily practical, certainly adds a fun and creative element to the capabilities of AI.
Another practical application involved a fine-tuned model that turned simple text descriptions into prompts suitable for DALLE or Midjourney. This could potentially simplify the process of generating prompts for other AI models.
Lastly, instead of training a small transformer to classify queries, a fine-tuned LLaMA was used as a supervised learner. This approach could potentially simplify the process of training AI models.
The "Anything But Wrappers" hackathon was a testament to the immense potential and versatility of AI. The event showcased how fine-tuning can be used to create practical solutions, automate tasks, and even add a dash of creativity to AI applications. As AI continues to evolve, we can only expect more groundbreaking developments in the future.
July 22, 2023
It's not always easy to understand the subtle nuances that exist between the business end and the technology end of projects. Where does one end and the other begin? How do they intertwine and interact? And most importantly, how can we navigate this spectrum to deliver the most effective solutions to our clients?
Welcome to the concept of the Business-Technology Gradient. This is a way to visualize and understand the multifaceted nature of web development projects. We break down the whole process into four categories: Business, Management, Design, and Technology. By understanding these categories and how they interact, we can better plan, manage, and execute our projects, creating solutions that are not only technically sound but also closely aligned with business goals. Let's dive in.
This category forms one end of the spectrum and represents high-level, strategic planning and decision-making activities. This is the stage where broad business objectives are determined, and overarching market strategies are developed. Key activities include defining the product-market fit, establishing unique selling propositions (USPs), and setting out marketing strategies. Sales planning and the analysis of return on investment (ROI) also fall under this umbrella. It's all about understanding the market, identifying opportunities, and positioning the business to leverage these opportunities. For example, in a software development context, this might involve identifying a market need for a particular kind of software solution, determining how your solution will be unique or better than existing options (the USP), and laying out strategies for reaching and convincing potential customers.
As the bridge between business and technology, this category involves coordinating and managing how the business's strategic goals are translated into technological solutions. This includes client and project management, where you'd coordinate with clients to understand their needs and manage the resources, timelines, and activities needed to deliver a solution. It also includes risk management, where you'd identify potential challenges or roadblocks in a project and plan how to avoid or mitigate them. This is about ensuring that business strategies are effectively carried out through technology.
This category is split into two distinct but related subcategories:
This involves the initial stages of designing a tech solution based on business needs. Key tasks include business analysis, where you'd examine the business's operations and identify how a tech solution could improve them; requirements gathering, where you'd define what the solution needs to do to provide value; and creating functional specifications, where you'd outline the specific features and functions the solution will have. This is about translating the high-level business strategy into a detailed plan for a technology solution.
This is where the tech solution starts to take shape. You'd be making key decisions about the software architecture, or how the solution will be structured and organized; designing the user interface to ensure a good user experience; and conducting entity-relationship modeling, or outlining how different elements of the solution will interact with each other. This stage sets the groundwork for the actual coding and building of the tech solution.
This is the other end of the spectrum, where the actual creation, implementation, and maintenance of the tech solution happen. This involves web development and programming, using specific tools and languages (like Ruby, JavaScript, SQL, etc.) to build the software. It also includes working with databases and servers (like PostgreSQL, Nginx, etc.), as well as DevOps activities to streamline the development process and ensure the solution is reliable and efficient. This is all about turning the design and plans from the earlier stages into a working, effective technology solution.
Here are examples of the types of activities that fall within each category of the business-technology gradient:
This list is a guide; specific tasks can differ based on unique project needs and business requirements.
In a nutshell, these categories cover the full range from strategic planning to detailed technical implementation, with each category building on the previous ones to deliver a cohesive, effective tech solution.