Connect with us

Sci-Tech

How LlamaIndex is ushering in the future of RAG for enterprises

Published

on


We want to hear from you! Take our quick AI survey and share your insights on the current state of AI, how you’re implementing it, and what you expect to see in the future. Learn More


Retrieval augmented generation (RAG) is an important technique that pulls from external knowledge bases to help improve the quality of large language model (LLM) outputs. It also provides transparency into model sources that humans can cross-check.

However, according to Jerry Liu, co-founder and CEO of LlamaIndex, basic RAG systems can have primitive interfaces and poor quality understanding and planning, lack function calling or tool use and are stateless (with no memory). Data silos only exacerbate this problem. Liu spoke during VB Transform in San Francisco yesterday.

This can make it difficult to productionize LLM apps at scale, due to accuracy issues, difficulties with scaling and too many required parameters (requiring deep-tech expertise).

This means that there are many questions RAG simply can’t answer.


Register to access VB Transform On-Demand

In-person passes for VB Transform 2024 are now sold out! Don’t miss out—register now for exclusive on-demand access available after the conference. Learn More


“RAG was really just the beginning,” Liu said onstage this week at VB Transform. Many core concepts of naive RAG are “kind of dumb” and make “very suboptimal decisions.”

LlamaIndex aims to transcend these challenges by offering a platform that helps developers quickly and simply build next-generation LLM-powered apps. The framework offers data extraction that turns unstructured and semi-structured data into uniform, programmatically accessible formats; RAG that answers queries across internal data through question-answer systems and chatbots; and autonomous agents, Liu explained.

Synchronizing data so it’s always fresh

It is critical to tie together all the different types of data within an enterprise, whether unstructured or structured, Liu noted. Multi-agent systems can then “tap into the wealth of heterogeneous data” that companies contain. 

“Any LLM application is only as good as your data,” said Liu. “If you don’t have good data quality, you’re not going to have good results.”

LlamaCloud — now available by waitlist — features advanced extract, transform load (ETL) capabilities. This allows developers to “synchronize data over time so it’s always fresh,” Liu explained. “When you ask a question, you’re guaranteed to have the relevant context, no matter how complex or high level that question is.”

LlamaIndex’s interface can handle questions both simple and complex, as well as high-level research tasks, and outputs could include short answers, structured outputs or even research reports, he said. 

The company’s LllamaParse is an advanced document parser specifically aimed at reducing LLM hallucinations. Liu said it has 500,000 monthly downloads and 14,000 unique users, and has processed more than 13 million pages. 

“LlamaParse is currently the best technology I have seen for parsing complex document structures for enterprise RAG pipelines,” said Dean Barr, applied AI lead at global investment firm The Carlyle Group. “Its ability to preserve nested tables, extract challenging spatial layouts and images is key to maintaining data integrity in advanced RAG and agentic model building.”

Liu explained that LlamaIndex’s platform has been used in financial analyst assistance, centralized internet search, analytics dashboards for sensor data and internal LLM application development platforms, and in industries including technology, consulting, financial services and healthcare

From simple agents to advanced, multi-agents

Importantly, LlamaIndex layers on agentic reasoning to help provide better query understanding, planning and tool use over different data interfaces, Liu explained. It also incorporates multiple agents that offer specialization and parallelization, and that help optimize cost and reduce latency. 

The issue with single-agent systems is that “the more stuff you try to cram into it, the more unreliable it becomes, even if the overall theoretical sophistication is higher,” said Liu. Also, single agents can’t solve infinite sets of tasks. “If you try to give an agent 10,000 tools, it doesn’t really do very well.”

Multi-agents help each agent specialize in a given task, he explained. It has systems-level benefits such as parallelization costs and latency.

“The idea is that by working together and communicating, you can solve even higher-level tasks,” said Liu. 



Source link
Continue Reading
Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *

Sci-Tech

Could powerful lasers power a working reactor?

Published

on


Damien Jemison A technician adjusts some of the optical equipment of the National Ignition Facility.Damien Jemison

The National Ignition Facility in California uses powerful lasers to spark fusion reactions

Deep under the Nevada desert in the 1980s the US conducted secret nuclear weapons research.

Among the experiments was an effort to see if nuclear fusion, the reaction which powers the sun, could be sparked on earth in a controlled setting.

The experiments were classified, but it was widely known among physicists that the results had been promising.

That knowledge caught the attention of two young graduate students working at the Los Alamos National Laboratory in the late 2000s, Conner Galloway and Alexander Valys.

The Los Alamos lab was originally set up in 1943 as a top-secret site to develop the first nuclear weapons. Located near Santa Fe, New Mexico it is now a US government research and development facility.

“When Alex and I learned about those tests at Los Alamos, our reaction was like ‘wow, inertial fusion has already worked!’. Laboratory-scale pellets were ignited, the details were classified, but enough was made public that we knew that ignition was achieved,” says Mr Galloway.

Nuclear fusion is the process of fusing hydrogen nuclei together, which produces immense amounts of energy. The reaction creates helium and not the long-lived radioactive waste of the fission process which is used in existing nuclear power stations.

If fusion can be harnessed, then it promises abundant electricity, generated without producing CO2.

Those tests in the 1980s led to the US government building the National Ignition Facility (NIF) in California, a project to see if nuclear fuel pellets could be ignited using a powerful laser.

After more than a decade of work, in late 2022 researchers at NIF made a breakthrough. Scientists conducted the first controlled fusion experiment to produce more energy from the reaction than that supplied by the lasers which sparked it.

A graphic showing the fusion reaction

While physicists around the world marvelled at that breakthrough, it had taken the scientists at NIF much longer than expected.

“They were energy starved,” says Mr Galloway.

He doesn’t mean they needed more snacks, instead the NIF laser was only just powerful enough to ignite the fuel pellet.

Mr Galloway and Mr Valys think that more powerful lasers will make it possible to build a working fusion reaction that can supply electricity to the power grid. To do that they founded Xcimer, based in Denver.

NIF had to make do with a laser that could pump out two megajoules of energy. Mr Galloway and Mr Valys are planning to experiment with lasers that can supply up to 20 megajoules of energy.

“We think 10 to 12 [megajoules] is the sweet spot for a commercial power plant,” says Mr Galloway.

Such a laser beam would hit the fuel capsule with a powerful punch. It would be like taking the energy of a 40-tonne articulated lorry travelling at 60mph and focussing it on the centimetre-sized capsule for a few billionths of a second.

More powerful lasers will allow Xcimer to use larger and simpler fuel capsules than NIF, which found it difficult to perfect them.

Xcimer Conner Galloway (left) and Alexander Valys stading in the big empty factory which will one day house their laser-based fusion project.Xcimer

Conner Galloway (left) and Alexander Valys founders of fusion firm Xcimer

Xcimer joins dozens of other organisations around the world trying to build a working fusion reactor.

There are two main approaches. Smashing a fuel pellet with lasers falls under the category of inertial confinement fusion.

The other way, known as magnetic confinement fusion, uses powerful magnets to trap a burning cloud of atoms called plasma.

Both approaches have daunting engineering challenges to overcome.

In particular, how do you extract the heat generated during fusion so you can do something useful with it, like drive a turbine to make electricity?

“I suppose my scepticism is, I haven’t yet even seen a persuasive conceptual diagram of how you manage the process of taking energy out while keeping the fusion reaction going,” says Prof Ian Lowe at Griffith University in Australia.

He has spent his long career working in energy research and policy. While Prof Lowe supports the development of fusion technology, he just argues that a working fusion reactor won’t come fast enough to help bring down CO2 emissions and tackle climate change.

“My concern is that even the most optimistic view is that we’d be lucky to have commercial fusion reactors by 2050. And long before then we need to have decarbonized the energy supply if we’re not going to melt the planet,” he says.

Another challenge is that the fusion reaction produces high energy particles that will degrade steel, or any other material that lines the reactor core.

Getty Images A truck exits the Los Alamos national laboratory, in New Mexico. A sign in the foreground says: "No Trespassing."Getty Images

Secret fusion tests were conducted at the Los Alamos National Laboratory in the 1980s

Those in the fusion industry don’t deny the engineering challenges, but feel they can be overcome.

Xcimer plans to use a “waterfall” of molten salt flowing around the fusion reaction to absorb the heat.

The founders are confident that they can fire the lasers and replace the fuel capsules (one every two seconds) while keeping that flow going.

The flow of molten salt will also be thick enough to absorb high energy particles that could potentially damage the reactor.

“We just have two relatively small laser beams coming in from either side [of the fuel pellet]. So you only need a gap in the flow big enough for those beams, and so you don’t have to turn off and turn on the entire flow,” says Mr Valys.

But how quickly can them make such a system work?

Xcimer plans to experiment with the lasers for two years, before building a target chamber, where they can target the fuel pellets.

The final stage would be the working reactor, which they hope would be plugged into the electricity grid in the mid-2030s.

To fund the first phase of their work, Xcimer has raised $100m (£77m) . The money will be used to build a facility in Denver and the prototype laser system.

Hundreds of millions dollars more will be needed to build a working reactor.

But for the founders of Xcimer, and other fusion start-ups, the prospect of cheap, carbon-free electricity is irresistible.

“You know, it’ll change the trajectory of what’s possible for humanity’s progress,” says Mr Valys.

More Technology of Business



Source link

Continue Reading

Sci-Tech

Apple banks on AI to boost sales of its new iPhone

Published

on


Getty Images People try out the new Apple iPhone 16 which has a button for the cameraGetty Images

With business slumping, Apple has been under pressure to show what it will offer buyers to jumpstart a new wave of iPhone sales.

On Monday, the technology giant revealed its hand – the iPhone 16 which has a camera button on the outside of the handset.

The button is an external clue to the changes Apple said it had made inside its latest smartphone, aimed at harnessing the latest in artificial intelligence (AI).

Apple’s chief executive Tim Cook said the upgrades would “push the boundaries of what a smartphone can do” but the firm has tough competition, as other brands have already integrated generative AI features into their handsets.

Apple’s share price fell during its “Glowtime” event, where it unveiled the iPhone 16 as well as other products, and ended the day flat. The company, worth $3 trillion, is facing concern that it is losing its edge in the burgeoning area of artificial intelligence.

Sales of the iPhone – Apple’s most important product which accounts for around half of its total sales – have stalled in recent months. They slipped by 1% over the nine months ended 29 June compared with a year earlier.

Apple said its new phones, which come with longer lasting batteries, more powerful chips and enhanced privacy features, were its first built specifically to handle AI and its new “Apple Intelligence” tools, many of which were announced in June.

Those include new tools for writing and creating new emojis as well incorporating OpenAI’s chatbot ChatGPT into Siri to help users with some queries and text generation requests.

On Monday, Apple also announced updates to its Apple Watch and its AirPod headphones, which will allow them to automatically drop the volume when users start in-person conversations and to decline calls with the shake of a head.

It said the Pro version of its AirPods would be able to be used as a “clinical grade” personal hearing aid for people with mild or moderate hearing loss.

The company said it was expecting marketing approval from regulators for the device “soon” and the feature would be available this autumn in more than 100 countries, including the US, Germany and Japan.

Previously, the company had a feature that allowed people to pair hearing aids with iPhones and other devices.

The products were rolled out at a glossy event where protestors gathered in a designated free speech area across the street, urging executives to ramp up efforts to protect children from dangerous content in the company’s App Store.

The protest featured a life-sized blow-up made to resemble Mr Cook.

Sales of the new range start in September, with prices for the iPhone16 starting at $799.

But the Apple Intelligence features are not set to be available on operating systems until October, starting in the US and heading to other countries in the following months. They will be available in the UK in December.

Ben Wood, chief analyst at the market research firm CCS Insight, said it was likely that many people would dismiss the company’s new camera control as a “glorified shutter button”.

But he said it offered “very significant” upgrades, including visual, AI-powered search and he came away from the presentation persuaded that Apple would win over customers.

“The combination of Apple Intelligence and new camera features on the iPhone 16 will help spur upgrades from loyal Apple customers,” he said. “Particularly as Apple is positioning this latest update as being a future-proof purchase for customers wanting to get Apple Intelligence features as they roll out over the next few years.”

EPA Apple's new AirPodsEPA

People will be able to decline phone calls with a shake of the head with Apple’s new AirPods

Apple has been slower than rivals Samsung and Google to bake generative AI features for photo editing, translation and web browsing into its devices.

Competitors are now building them into folding, flipping and even tri-folding smartphones.

Pre-orders for Huawei’s new tri-fold phone, the Mate XT, reportedly hit more than three million on Monday.

Gartner analyst Annette Zimmermann said because Apple was rolling out AI-ready smartphones later than rivals, it was “critical” they deliver.

She warned that rolling the features out before they were ready could risk their reputation or prompt sales losses.



Source link

Continue Reading

Sci-Tech

Google’s lucrative ad tech business goes on trial

Published

on


The US government is taking aim at the engine of Google’s immense wealth – its extremely lucrative ad tech business.

A trial beginning on Monday will hear the Department of Justice’s case that the search engine’s parent company Alphabet illegally operates a monopoly in the market.

The company earned more than $200 billion (£152bn) last year through the placing and selling of ads seen by internet users.

Alphabet has argued its success is due to the “effectiveness” of its services – but prosecutors say it has used its market dominance to stifle rivals.

“It is a really important industry that grabs billions of consumer dollars every year,” said Laura Phillips-Sawyer, a professor at the University of Georgia School of Law.

“I think all consumers have an interest in this litigation.”

It is the second major antitrust case the tech giant has faced in the US.

In August a judge ruled its dominance of search was illegal, with the penalties Google and Alphabet will face as a result of that decision so far unclear.

According to the lawsuit filed by the Department of Justice (DoJ) and a coalition of states in 2023, Google dominates the digital ad marketplace and has leveraged its market power to stifle innovation and competition.

Google meanwhile contends it is just one of several hundred companies that facilitate the placement of digital ads in front of consumers.

It argues that competition in the digital ad space is growing, not contracting – citing increased ad growth and revenues for companies such as Apple, Amazon and TikTok as proof in a blog post responding to the DoJ’s lawsuit in 2023.

Both sides will present their cases to US District Judge Leonie Brinkema, who is expected to deliver a verdict.

The bench trial comes on the heels of a landmark decision last month in a different monopoly case brought by the Justice Department against Google.

Judge Amit Mehta ruled that Google acted illegally to squelch competition in its online search business.

“Google is a monopolist, and it has acted as one to maintain its monopoly,” he wrote.

During last year’s trial, Google said it dominated online search because it had a better product.

And the company is seemingly deploying a similar defence in the ad tech case.

When asked for a statement, it referred the BBC to its 2023 blog post, in which it states that “no-one is forced to use our advertising technologies – they choose to use them because they’re effective.”

Judge Mehta held a status conference on Friday as he begins the process of deciding on remedies for Google’s conduct.

“The DoJ clearly had a big win, and they’re going to ride that momentum,” Dan Ives, managing director at Wedbush Securities, told the BBC.

He said he expects those remedies to involve “business model tweaks, not a breakup” of the company.

Meanwhile, in Justice Brinkema’s courtroom, the arcane process that governs advertising technology could make the DoJ’s attempts to prove its case an uphill climb.

“We all use search. We all intuitively understand that product,” said Rebecca Haw Allensworth, an antitrust professor at Vanderbilt University Law School.

By comparison, advertising technology is “so complex that I think that’s going to be a real challenge for the government to make a clear, simple monopolisation argument here.”

The US is not the only country where regulators are unhappy with Google’s ad tech business.

On Friday, the UK Competition and Markets Authority said it believed Google was abusing its dominance in the ad tech industry, according to the findings of its initial investigation.

It said it found that Google used anti-competitive practices to dominate the market for online advertising technology – and the potentially unlawful behaviour could be harming thousands of UK publishers and advertisers.

A Google representative said the decision was based on a “flawed” understanding of the ad tech sector.



Source link

Continue Reading
Advertisement

Trending

paribahis bahsegel bahsegel bahsegel bahsegel resmi adresi

Copyright © 2024 World Daily Info. Powered by Columba Ventures Co. Ltd.