Last week, many important figures in the chip field gathered in San Francisco to participate in ISSCC 2023, the world’s most important chip field conference. This year is the 70th ISSCC conference, and its theme is “Building on 70 years of innovation in solid-state circuit design”. The past 70 years have been 70 years of rapid progress in the semiconductor chip industry, but today, where is the next opportunity for innovation in the chip field? In this year’s ISSCC keynote speech (Plenary) and papers published at the conference, we believe that the core keyword is “system innovation” (system innovation).
In the keynote speech “Innovation for the next decade of compute efficiency” (Innovation for the next decade of computing efficiency) brought by AMD’s CEO Lisa Su, Su mentioned the rapid advancement of AI applications and the demands it brings to chips. With the gradual popularity of the large language model (LLM) represented by ChatGPT, the number of parameters of the AI model has increased exponentially, and correspondingly, the demand for computing chips and memory has also increased rapidly, but the efficiency of the current chip is not enough to meet the needs of the model. demand. According to the current rule that computing efficiency increases by 2.2 times every two years, it is estimated that by 2035, a supercomputer will need 500mW of power, which is equivalent to the power that half a nuclear power plant can generate. Obviously, in order to meet such computing requirements, the improvement of computing efficiency needs to be far more than 2.2 times every two years, and in order to achieve such efficiency improvement, system-level innovation is one of the most critical ideas.
In another keynote speech delivered by IMEC/CEA Leti/Fraunhofer, the three most famous semiconductor research institutions in Europe, system-level innovation is also a core keyword. The speech mentioned that as the semiconductor process is gradually approaching the physical limit, new application requirements for chips must also be met from the system level, and mentioned the next generation of smart cars and AI as two particularly need chips from the system level Only innovation can support the core application of its new requirements.
What is system-level innovation?
We have seen that system-level innovation was repeatedly mentioned in the most important keynote speech of this year’s ISSCC, so what is system-level innovation? The current consensus is that system-level innovation refers to the collaborative design of multiple links in the upstream and downstream of the overall design to improve performance. At the system level, upstream technology includes application software, algorithms, system architecture, component requirements, etc., and these upstream requirements will eventually be reflected in chip requirements, including chip design, semiconductor device design, and semiconductor process design. wait. In past designs, performance improvement was often only reflected in one dimension. For example, the success criterion of circuit design was whether the clock frequency of digital circuits could run fast enough, whether the bandwidth of analog circuits could be made larger, and so on. In system-level innovation, performance improvement is not only achieved by design improvement at the technical level, but more importantly, it is achieved by upstream application, algorithm and system innovation and by circuit, device and process level design to meet these requirements to complete the overall performance. improvement.
Lisa Su gave a classic case of system-level innovation in her speech, which is to use some innovative number systems (such as 16-bit floating-point number BF16 or 8-bit floating-point number FP8) through optimization at the algorithm level of artificial intelligence models. By supporting the optimization of these algorithms at the circuit level, the efficiency of the calculation level can be improved by an order of magnitude. Compared with the traditional 32-bit floating-point number (FP32), the new BF16 can increase the calculation efficiency by more than 10 times, while FP8 can increase the calculation efficiency by as much as 30 times. This is also the power of system-level innovation: According to traditional thinking, if the circuit design only stays in the circuit dimension and only considers how to further optimize the efficiency of the FP32 computing unit, it will be difficult to achieve an order of magnitude improvement in efficiency anyway. This is why when new applications put forward very aggressive performance requirements for chip performance, the chip industry needs to meet such requirements from a system-level perspective.
Semiconductor technology as the backbone of system-level innovation
As mentioned earlier, the design of semiconductor chips needs to consider system-level innovations in order to meet the needs of new applications for chip performance improvements. From another perspective, in such system-level innovation, some important semiconductor technologies will become the core support. Therefore, these semiconductor technologies will become extremely important when system-level innovation becomes the driving force for the evolution of core technologies today.
We believe that system-level innovation means that the overall chip system will become more complex: complexity here means that there will be more chips in the system (this also includes splitting a large chip into multiple small chip particles), There is therefore a need to be able to support such multi-chip systems in a flexible manner, while providing significant improvements in performance and efficiency. Once semiconductor technology can provide such a platform to support such a complex system, then system-level design will have a larger design space, thus providing strong support for system-level innovation. The semiconductor technology that provides efficient support for such complex chip systems is above all advanced packaging technology. Using advanced packaging technology, complex chip systems can be integrated in a package in an efficient form, and provide very high communication bandwidth, so it can provide support for system-on-chip innovation. For example, advanced packaging can integrate the traditional on-chip cache (cache) and the processor chip in the form of chips, which greatly reduces the limitation of the semiconductor process on the cache capacity, thus providing a larger design space for system design , and such a design (3D V-Cache) has been proven to significantly improve the performance of the processor and AMD has been used in the product. In the future, we can expect to see more designs.
Another important technology related to it is the IO interface technology, which can further improve the transmission speed and efficiency of the interconnection between chips. In Lisa Su’s keynote speech, in addition to advanced packaging that can improve the bandwidth of chip data interconnection in the package, another important dimension is medium and long-distance data interconnection, which is used between multiple chip systems and between multiple servers Transfer data at high speed and efficiently. The evolution of such an IO interface technology can greatly reduce the communication overhead between various components in the system, thereby realizing a highly efficient complex chip system. In terms of mid-distance (between chips) data interconnection, ultra-high-speed SerDes technology will become an important supporting technology in the future. At this year’s ISSCC, Nvidia also released its NvLink-C2C technology for interconnection between chip packages, which can achieve up to 40Gbps/pin, and has been used in its next-generation GPU system (Hopper). It can be seen that the data interconnection technology The importance has become the consensus of chip giants.
In terms of long-distance interconnection, optical interconnection technology is expected to become the main direction of technological evolution, and its transmission distance can reach 10m to 2km, and the transmission energy cost can be very low (<1pJ/bit). In this field, optical transmission transceiver modules and computing chips need to be tightly integrated (for example, using co-packaged optics technology), and we expect rapid technological evolution in the future.
In summary, we believe that system-level innovation has become the main driving force for performance improvement in the chip industry. System-level innovation includes not only considering the circuit design upstream of the system (such as algorithms and applications) to achieve an order of magnitude improvement in performance and efficiency, but also including new semiconductor technology platforms (such as advanced packaging and IO technology) to provide more for system design. design space. The new opportunities provided by system-level innovations for chips are worth looking forward to.
Post time: Mar-08-2023