Home Nvidia unveils AI chatbot ‘Chat with RTX’ for local PCs

Nvidia unveils AI chatbot ‘Chat with RTX’ for local PCs

Nvidia has released an early version of its Chat with RTX, which is a demo that lets users personalize a chatbot with their own content on Windows PCs.

The custom generative AI is now free to download, and it requires an RTX 30 or 40-series GPU with at least 8GB of VRAM. This means users will have the capability to upload their own documents in order to create summaries and receive relevant answers based on their personal data.

Chat with RTX can apparently let you search YouTube URLs and its transcripts for specific mentions or summarize an entire video.

Users will be able connect local files on a PC as a dataset to an open-source large language model like Mistral or Llama 2, enabling queries for relevant answers. According to Nvidia’s blog, Chat with RTX “uses retrieval-augmented generation (RAG), NVIDIA TensorRT-LLM software and NVIDIA RTX acceleration to bring generative AI capabilities to local, GeForce-powered Windows PCs.”

RAG is essentially an assistant that searches through the data, especially if the model set is particularly large. The tool supports various file formats, including .txt, .pdf, .doc/.docx and .xml.

The company, which became the fourth most valuable U.S. corporation this week, said by utilizing the power of local GeForce-equipped Windows PCs, users can enhance their experience and take advantage of generative AI with unparalleled speed and privacy.

“Rather than relying on cloud-based LLM services, Chat with RTX lets users process sensitive data on a local PC without the need to share it with a third party or have an internet connection,” it added.

Nvidia also invited developers to discover how RTX GPUs can potentially speed up large language models by consulting the TensorRT-LLM RAG developer reference project on GitHub.

Nvidia’s ongoing success

In recent months, the company has experienced an extraordinary surge in its value. In December, Nvidia’s stock value tripled, outperforming every other company in the S&P 500. Its chips have proved crucial at a time when there is a global shortage.

However, there has been controversy over its sale of chips to Chinese military bodies and state-affiliated groups, despite a U.S. ban on the export of the commodity to China.

Featured image: Canva

About ReadWrite’s Editorial Process

The ReadWrite Editorial policy involves closely monitoring the tech industry for major developments, new product launches, AI breakthroughs, video game releases and other newsworthy events. Editors assign relevant stories to staff writers or freelance contributors with expertise in each particular topic area. Before publication, articles go through a rigorous round of editing for accuracy, clarity, and to ensure adherence to ReadWrite's style guidelines.

Suswati Basu
Tech journalist

Suswati Basu is a multilingual, award-winning editor and the founder of the intersectional literature channel, How To Be Books. She was shortlisted for the Guardian Mary Stott Prize and longlisted for the Guardian International Development Journalism Award. With 18 years of experience in the media industry, Suswati has held significant roles such as head of audience and deputy editor for NationalWorld news, digital editor for Channel 4 News and ITV News. She has also contributed to the Guardian and received training at the BBC As an audience, trends, and SEO specialist, she has participated in panel events alongside Google. Her…

Get the biggest tech headlines of the day delivered to your inbox

    By signing up, you agree to our Terms and Privacy Policy. Unsubscribe anytime.

    Tech News

    Explore the latest in tech with our Tech News. We cut through the noise for concise, relevant updates, keeping you informed about the rapidly evolving tech landscape with curated content that separates signal from noise.

    In-Depth Tech Stories

    Explore tech impact in In-Depth Stories. Narrative data journalism offers comprehensive analyses, revealing stories behind data. Understand industry trends for a deeper perspective on tech's intricate relationships with society.

    Expert Reviews

    Empower decisions with Expert Reviews, merging industry expertise and insightful analysis. Delve into tech intricacies, get the best deals, and stay ahead with our trustworthy guide to navigating the ever-changing tech market.