Your Tech Story

supercomputers

supercomputers

Are Google’s AI supercomputers faster than Nvidia’s?

Leave a Comment / Story, Tech Story / By

As powerful models for machine learning continue to be the hottest topic in the tech business, Google released information about one of the company’s AI supercomputers on Wednesday, claiming it is quicker and more effective than rival Nvidia systems.

Tensor Processing Units, or TPUs, are artificial intelligence (AI) chips that Google has been developing and utilizing since 2016. Nvidia currently holds a 90% share of the overall market for AI training models and deployment.

supercomputers
Image Source: techzine.eu

As a leading innovator in AI, Google has produced several of the most significant developments in the area during the past ten years. However, some people think the company has lagged behind in commercializing its ideas, and internally, the corporation has been rushing to develop items to show it hasn’t wasted its lead, creating a “code red” situation.

Also Read: What’s Next for Users as Google Now Launcher Shuts Down?

A large number of supercomputers and a lot of processors must work simultaneously to train models, with the computers operating nonstop for weeks or months, as is the case with AI models and products like Google’s Bard or OpenAI’s ChatGPT, which are powered by Nvidia’s A100 chips.

On Tuesday, Google announced that it has developed a system with more than 4,000 TPUs connected to specialized parts intended to operate and train AI models. It has been in operation since 2020 and has been used for 50 days to train Google’s PaLM model, which challenges OpenAI’s GPT model.

The Google researchers claimed that the TPU-based supercomputer, known as TPU v4, is “1.2x-1.7x faster and uses 1.3x-1.9x less power than the Nvidia A100.” The researchers said, “The performance, scalability, and availability make TPU v4 supercomputers the workhorses of large language models.”

The H100, the most recent Nvidia AI chip, was not compared to Google’s TPU results, however, because the H100 is more modern and was manufactured using more sophisticated manufacturing techniques, according to Google researchers.

Nvidia CEO Jensen Huang said that the findings for the company’s most current chip, the H100, were noticeably faster than those for the previous generation. findings and rankings from an industry-wide AI chip test called MLperf were published on Wednesday.

Given the high cost of the significant computing power required for AI, many in the sector are concentrating on creating new processors, hardware elements like optical links, or software innovations that will lower the required computing power.

Also Read: Can we use nearby share between Android and Windows?

The computational demands of AI also benefit cloud service providers like Google, Microsoft, and Amazon, who may rent out computer processing on an hourly basis and give startup companies credits or computing time to foster business partnerships. For instance, Google claimed that their TPU chips were used to train the AI image generator Midjourney.

ai supercomputer

Nvidia and Microsoft Collaborate To Build AI Supercomputers

Leave a Comment / News, Tech News / By

Nvidia and Microsoft are working together over a “multi-year collaboration” to build “one of the most powerful AI supercomputer in the world,” which will be capable of handling the large processing workloads needed to teach and scale AI.

ai supercomputer
Image Source: ciobulletin.com

According to the reports, the AI supercomputer will be driven by Microsoft Azure’s cutting-edge supercomputing technology along with Nvidia GPUs, networking, and a comprehensive stack of AI software to support businesses in training, deploying, and scaling AI, including big, cutting-edge models. 

NVIDIA’s A100 and H100 GPUs will be part of the array, coupled with their Quantum-2 400Gb/s Infiniband networking technology. In particular, this would be the first public cloud to feature NVIDIA’s cutting-edge AI tech stack, allowing businesses to train and use AI on a large scale.

Manuvir Das, VP of enterprise computing at Nvidia, noted, “AI technology advances as well as industry adoption are accelerating. The breakthrough of foundation models has triggered a tidal wave of research, fostered new startups and enabled new enterprise applications. Our collaboration with Microsoft will provide researchers and companies with state-of-the-art AI infrastructure and software to capitalise on the transformative power of AI.”

NVIDIA will work with Azure to research and accelerate advancements in generative AI, a rapidly developing field of artificial intelligence where foundational models like the Megatron Turing NLG 530B serve as the basis for unmonitored, self-learning algorithms to generate new text, digital images, codes, audio, or video.

Additionally, the companies will work together to improve Microsoft’s DeepSpeed deep learning optimization tool. Azure enterprise clients will have access to NVIDIA’s full suite of AI processes and software development tools that have been tailored for Azure. In order to speed up transformer-based models used for huge language models, generative AI, and generating code, among other uses, Microsoft DeepSpeed will make use of the NVIDIA H100 Transformer Engine.

With twice the capacity of 16-bit operations, this technology uses DeepSpeed’s 8-bit floating point accuracy capabilities to significantly speed up AI calculations for transformers.

Verge reports that due to the recent quick expansion of these AI models, there has been a major increase in the demand for robust computer infrastructure.

The partnership is intriguing for several reasons, but notably, because it makes one of the largest computing clusters accessible to companies. Due to optimizations made in the newest ‘Hopper’ generation of NVIDIA GPUs, this will not only enable companies to train and implement AI at a level that was previously impractically pricey to do but also allow them to do it at a significant level of efficiency.

Although NVIDIA has a supercomputer of its own, the collaboration demonstrates that they are aware of the enormous computational demands placed on them by contemporary algorithms. This development represents a collaboration between two of the largest organizations in the AI industry.

Microsoft has experience in this area, as seen by its relationship with OpenAI and dedication to the development of ethical and safe AI. Contrarily, NVIDIA has been a pillar of AI development and research for the past ten years thanks to its potent GPUs and supporting tech stack, which includes CUDA and Tensor cores.

Control Data Corporation

Control Data Corporation – The Story of One of the Pioneers of Mainframe and Supercomputers.

Leave a Comment / Story, Tech Story / By

There have been cases where companies do exceptionally well and stand as a tough competitor for their rivals at a time, and when the time changes, it becomes difficult for them to keep up with the changing trends, and they fall bad. The same thing happened with CDC (Control Data Corporation), a company founded in 1957, termed as one of the pioneers of the mainframe and the supercomputers. In the beginning years of the company, it easily became a part of the top ten leading computer manufacturing companies in the US. But the changing technologies made the company suffer huge losses, and it had to close most of its operations.

Foundation of Control Data Corporation

A couple of software engineers from a team (later became a company named Sperry) that worked for the US military in World War II established Control Data Corporation in September 1957. The company started its operations from an old warehouse in Minneapolis. William Norris became the first CEO, and Seymour Cray was the first chief designer of the company.
With enough experience, CDC started to build and ship the subsystems, including drum memory systems. The next year, the company built its flagship product, the CDC Little Character, one of the most successful products of CDC. In the following years, CDC also built products for the US Navy. The company is also allegedly credited for building one of the first minicomputers, i.e. the CDC 160A, a 12-bit version of 1604. The CDC 160A was the first computer to have an architecture of a standard office desk.
CDC 3000 series was CDC’s next famous series of the 60s. During the mid-60s, Seymour Cray, Jim Thornton, and Dean Roush were already working on a new computer design separately to improve the CDC 1604 and bring a more compact design computer to the market. The result was CDC 6600, a ten times faster machine that brought about $8 million to the company profits, and over 100 machines were sold. The CDC 6600 brought much-needed fame to CDC, and IBM started to consider the former as its rival.

Control Data Corporation
Image source: wikimedia.org

During the same time, to grow and compete with companies like IBM, CDC started to acquire other small companies. As a result, CDC was now also developing peripheral products like tape transport. The peripheral lineup of CDC was making cheaper yet faster peripheral devices as compared to its rivals. CDC became one of the leading companies to supply hard disk drives through its Magnetic Peripheral Inc (Imprimis Technology) division. CDC is one among the three developers of the universal Advanced Technology Attachment (ATA) interface along with Compaq and Western Digital.
In the 70s, CDC also entered into the OCR system development business to replace the punched card technology and acquired Rabinow Engineering (one of the pioneers of OCR technology). But later, CDC had to close that business.
The next products from CDC were CDC 7600 and CDC 8600, which were even faster, but could not reach the expectation. In 1975, CDC introduced the STAR-100, a supercomputer that had its name registered in the Guinness Book of World Records for being the “most powerful and fastest computer”.
During the late 70s, CDC was selling CDC 6600 and 7600 under its CDC Cyber lineup. The company’s other ambitious and successful projects were Cyber 205, CYBER PLUS, Cyber 80, etc.

The ‘Fall’, and Formation of Ceridian

The 80s was the most challenging decade for CDC. It was selling the machines but was unable to produce competitive designs as per the requirements. With the rise in competition, the company started to sell some of its business bit-by-bit, starting with PathLab Laboratory Information System in 1987, followed by VTC, Ticketron, and CDC in 1992. CDC was then left with its service business, which is renamed Ceridian, a company that is still operational.
Ceridian, today, is following its legacy and is working as one of the leaders in the field of IT outsourcing, mainly as a human resource company. Ceridian is active in five major countries, i.e., the USA, Canada, Europe, Australia, and Mauritius. It is a publicly traded company and has its headquarters based in Minneapolis, Minnesota, USA.

The Key People in the Formation of CDC

William Norris was the first CEO and one of the founders of CDC. He was born on 14 July 1911 in Red Cloud, Nebraska. He went to the University of Nebraska and started his career as an X-Ray equipment seller for the Westinghouse Corporation in Chicago.
Later, Norris joined the US Navy as a codebreaker and became the lieutenant commander in the next few years. After World War II ended, he along with Howard Engstrom and other US Navy cryptographers, formed the Engineering Research Associates (ERA) in January 1946, a division that worked for the US Navy. After multiple restructuring of ERA, Norris with a few other employees left the company and established Control Data Corporation in 1957. Norris was unanimously selected as the president and the first CEO of CDC.