<\/p>\n
Etched is making waves<\/a> in the artificial intelligence hardware space with its revolutionary new AI accelerator chip. The Silicon Valley startup, founded in 2022 by Harvard dropouts Gavin Uberti and Chris Zhu, has developed a custom application-specific integrated circuit (ASIC) called Sohu that is purpose-built to run transformer models \u2013 the architecture behind today\u2019s most advanced AI systems.<\/p>\n Etched claims its Sohu chip can process AI workloads up to 20 times faster than Nvidia\u2019s top-of-the-line GPUs while using significantly less power. With $120 million<\/a> in fresh funding and partnerships with major cloud providers, Etched is positioning itself as a formidable challenger to Nvidia\u2019s dominance in AI chips.<\/p>\n Primary Venture Partners and Positive Sum Ventures led the funding round, which included participation from high-profile investors like Peter Thiel, Github CEO Thomas Dohmke, and former Coinbase CTO Balaji Srinivasan. As transformer models continue to drive breakthroughs in generative AI, Etched\u2019s specialized hardware could reshape the landscape of AI computing.<\/p>\n Etched\u2019s approach targets the complexities of GPUs and TPUs, particularly the need to handle arbitrary CUDA and PyTorch code, which demands sophisticated compilers. While other AI chip developers like AMD, Intel, and AWS have invested billions into software development with limited success, Etched is narrowing its focus. By exclusively running transformers, Etched can streamline software development for these models.<\/p>\n Most AI companies use transformer-specific inference libraries such as TensorRT-LLM, vLLM, or HuggingFace\u2019s TGI. Although somewhat inflexible, these frameworks suffice for most needs because transformer models across different applications\u2014text, image, or video\u2014are fundamentally similar. This allows users to adjust model hyperparameters without altering the core model code. However, the most prominent AI labs often require custom solutions, employing engineers to optimize GPU kernels meticulously.<\/p>\n Etched aims to eliminate the need for reverse engineering by making its entire software stack open source, from drivers to kernels. This openness allows engineers to implement custom transformer layers as needed, enhancing flexibility and innovation.<\/p>\n Etched\u2019s approach to AI hardware is comparable to the advancements seen with Groq\u2019s LPU Inference Engine. Groq\u2019s LPU, a dedicated Language Processing Unit,<\/a> has set new benchmarks in processing efficiency for large language models, surpassing traditional GPUs in specific tasks. According to ArtificialAnalysis.ai, Groq\u2019s LPU achieved a throughput of 241 tokens per second with Meta AI\u2019s Llama 2-70b model, demonstrating its capability to process large volumes of more straightforward data more efficiently than other solutions.<\/p>\n This level of performance spotlights the potential for specialized AI hardware to revolutionize the field by offering faster and more efficient processing capabilities tailored to specific AI workloads. Etched claims its ASIC achieves as many as 500,000 tokens per token with its hardware, dwarfing Groq\u2019s performance.<\/p>\n The introduction of ASICs for Bitcoin mining<\/a> marked a revolutionary shift in the landscape, fundamentally altering the network dynamics. When ASICs were first introduced in 2013, they represented a quantum leap in mining efficiency compared to the CPUs and GPUs that had previously dominated the field. This transition profoundly impacted Bitcoin\u2019s ecosystem, dramatically increasing the network\u2019s overall hash rate and, consequently, its security.<\/p>\n ASICs, being purpose-built for Bitcoin mining, offered unprecedented computational power and energy efficiency, quickly rendering CPU and GPU mining obsolete for Bitcoin. This shift led to a rapid centralization of mining power, as only those with access to ASIC hardware could profitably mine Bitcoin. The ASIC era ushered in industrial-scale mining operations<\/a>, transforming Bitcoin mining from a hobby accessible to individual enthusiasts into a highly competitive, capital-intensive industry.<\/p>\n Etched\u2019s vision began in 2022 when AI technologies like ChatGPT were not yet prevalent, and image and video generation models primarily relied on U-Nets and CNNs. Since then, transformers have become the dominant architecture across various AI domains, validating Etched\u2019s strategic focus.<\/p>\n The company is rapidly advancing toward one of the quickest chip launches in history. It has attracted top talent from major AI chip projects, partnered with TSMC for their advanced 4nm process, and secured essential resources such as HBM and server supply to support initial production. Early customers have already committed tens of millions of dollars to Etched\u2019s hardware.<\/p>\n This rapid progress could dramatically accelerate AI capabilities. For instance, AI models could become 20 times faster and cheaper overnight. Current limitations could be drastically reduced, such as the slow response times of models like Gemini or the high costs and long processing times of coding agents. Real-time applications, from video generation to AI-driven conversations, could become feasible, addressing the current bottlenecks<\/a> faced even by leading AI firms like OpenAI during peak usage periods.<\/p>\n Etched\u2019s advancements promise to make real-time video, calls, agents, and search a reality, fundamentally transforming AI capabilities and their integration into everyday applications.<\/p>\nEtched transformer ASICS for LLMs<\/h2>\n
ASICs changed the game for Bitcoin; will they do the same for AI?<\/h2>\n
Etched history and development<\/h2>\n
Mentioned in this article<\/h6>\n