The best LLM tools for software development

This post provides an overview of the various LLM-based generative development tools available out there. Are we missing an AI tool for coding? Let us know!

Large Language Models are now advanced enough to actually provide valuable help in software development tasks. There are a range of AI-based coding assistants and code generation tools.

If you want to use an AI model to help out in your software development workflow (whether it’s coding, documentation, or other tasks), you’ll want first to consider the security of your data. Explicitly define if you’re OK with:

• Information and code leaving your machine, meaning you have to trust a third party that they won’t (un-)willingly disclose any of it
• LLMs being trained or fine-tuned using your prompts (usually also including your code).

Plain old ChatGPT is still the most popular tool for developers, with GitHub Copilot and Google Gemini making it into the top 3. Yet there are a range of tools out there that can prove to be helpful in accelerating your development workflow.

Want to find out which base models are best suited for your software development tasks? Check out our own DevQualityEval. For an up-to-date ranking of the top-performing (research and production-grade) LLM-based systems resolving real-world GitHub issues, check out the SWE-bench leaderboard.

Below you’ll find a list of the most popular AI software development tools, categorized by use case and listed in alphabetical order.

In-IDE coding assistant tools

In-IDE tools are popular because they help right where your spend your time working with code anyway: your development environment.

Amazon Q Developer

Amazon Q Developer now includes what you may have known as Amazon CodeWhisperer in the past. It’s ranking highly on the SWE Bench leaderboard, taking the 6th place at the time of writing.

Q Developer is powered by Amazon Bedrock which is a separate product that makes foundation models accessible via an API. The tool generates code based on comments in natural language and existing code. Amazon Q Developer can predict and complete code blocks while you are programming. You can also give natural language prompts to the LLM with inline comments in your code. Amazon Q Developer will then generate code to that effect. Single-click actions let you copy that code for use in your source code file.

Customizations let you fine-tune Amazon Q Developer using your codebase. The tool is available for JetBrains IDEs, VS Code, Visual Studio, CLI, and your console.

AppMap Navie

AppMap Navie is another context-aware AI coding assistant that focuses on providing insight on your codebase. It is available as an extension to VS Code, a plugin for JetBrains IDEs (JetBrains, PyCharm, IntelliJ, WebStorm, and RubyMine), and the tool also has an enterprise edition. AppMap Navie supports OpenAI LLMs via Azure or the AppMap hosted proxy, and local models via Ollama & Anyscale. Navie analyzes your codebase to provide personalized suggestions, and has a chat window for you to ask questions. At the time of writing, AppMap Navie (used with GPT-4o) is 5th on the SWE-bench leaderboard.

AskCodi

AskCodi helps you generate, refactor, and fix code, and it can also transpile your code across languages. The AskCodi website provides access to a huge variety of apps such as a CI/CD pipeline writer, a Makefile writer, a Query writer, a Regex generator, a Schema resolver, a Complexity analyzer, etc.

AskCodi offers RAG-based, improved suggestions based on custom codebases. The tool supports models by OpenAI, Anthropic, Meta, and more including GPT 4o, Claude 3.5 Sonnet, Mistral Large, Claude 3 Opus, and others. AskCodi comes as an extension or plugin for Visual Studio Code, Sublime Text, and JetBrains IDEs, but a web interface is also available at the provider’s website.

Codeium

Not to be confused with CodiumAI, Codeium is a free LLM-based tool that uses proprietary models to provide (single and multi-line) autocomplete and chat functionality through full awareness of your repository context. Autocompletion suggestions are provided inline, and Codeium can also perform tasks on code, with suggestions not just copied, but directly applied with one click. It offers an intelligent search feature that lets you find files and code that are related to your intent.

Codeium supports 40+ editors (VS Code, JetBrains IDEs, Vim, Neovim, Visual Studio, Eclipse, etc) and 70+ languages (Javascript, Python, Typescript, PHP, Go, Java, C, C++, Rust, Ruby, etc). There’s even a Chrome extension to be used with Colab.

CodiumAI

Not to be confused with Codeium, CodiumAI is another code generation tool that combines code completion, real-time coding assistance, the generation of detailed task implementation plans, and code review. It’s powered by a proprietary model based on OpenAI’s GPT-4.

Through one of its (open-source) components, CodiumAI can also help with documentation, test generation, and PRs including their reviews. The CodiumAI IDE plugin also provides code analysis and test generation capabilities.

CodiumAI supports all programming languages for test generation, with the primary focus being on Python, JavaScript, TypeScript, and Java.

Cody by Sourcegraph

Using your codebase as context, Cody generates autocomplete suggestions (single lines to whole functions). You’ll communicate with the LLM of your choice via natural language to prompt the model to generate, test, or fix code (you can also provide prompts as inline comments). Cody can help you navigate the codebase with a code search feature and detailed code intelligence (references, code owners, trace history, etc).

The tool supports a range of models by OpenAI, Anthropic, Mistral, Google Gemini, and Fireworks.ai, and is available for, IntelliJ editors, VS Code, Neovim, as well as a web application. It can connect to codehosts like GitHub or GitLab, too.

Continue

Continue is an open-source AI code assistant that lets you connect, its developers claim, “any model”, with some of the most common LLMs recommended for use (LLama 3, DeepSeek Coder v2, Claude 3, GPT-4o, Gemini Pro, StarCoder 2 etc).

This tool lets you build custom autocomplete and chat experiences in your IDE with autocomplete for single lines or larger blocks of code. It provides a codebase retrieval feature, and chat functionality to ask questions about your codebase. A smart idea from Continue is that they provide prompt files to use as templates for tasks like writing a unit test using Jest. VS Code users can also create custom quick actions for single-click access to LLM responses. Continue works with VS Code and JetBrains IDEs.

GitHub Copilot

GitHub Copilot provides coding assistance via code completion and code generation right in your IDE. It works by offering inline suggestions as you write code.

The tool is powered by LLMs of GitHub, OpenAI, and Microsoft. It supports popular programming languages but works best for languages with ample training data (e.g. JavaScript). You can use GitHub Copilot as an IDE extension in a range of IDEs: VS Code, JetBrains IDEs, and Neovim. There is even a “voice coding” extension for VS Code that allows you to talk to GitHub Copilot.

Note that you can fine-tune the model to get a custom and private Copilot that’s based on your own repositories, but there’s currently a waitlist for this feature.

Pythagora

Pythagora is powered by GPT Pilot and GPT-4 and can write full features, debug code, explain issues, and review code in response to natural language prompts. It can write automated tests, fix its own code by reading errors from terminal, and process external documentation to have more context for your code. The tool tracks your entire codebase with Git.

Better still, Pythagora is meant to be used for building applications from the ground up. You can start by describing your project so the AI can suggest formulations for specifications. You can then take these specs to have the LLM create the architecture for your project, break down the project into tasks, and create a development plan. Then, the developer agent takes over to start implementing the project task by task, with a reviewer agent checking all code changes. The tool will then prompt you to test the application.

Pythagora offers one-click deployment to the cloud and works best for web app projects that use Node.js, Express, and MongoDB with JavaScript or TypeScript. The tool is available as an extension to VS Code.

TabbyML

TabbyML is an open-source solution that lets you set up a self-hosted LLM-powered code completion server. You can use and combine models of your choice from a range of available LLMs including versions of StarCoder, CodeLlama, DeepseekCoder, CodeGemma, CodeQwen, Codestral, etc. TabbyML runs locally and even allows you to load models from a local directory as long as it follows the specifications described in Tabby’s Model Specification.

Like most others, the tool provides inline code completion (multi-line) and offers an “explain code” feature. Tabby can also generate unit tests and documentation for your code. It works as an extension to VS Code, IntelliJ IDEs, and VIM/NeoVIM.

Tabnine

Tabnine provides context-aware generated code suggestions based on your existing code. Suggestions are provided in a separate tab on your activity bar - think of it as chatting with the LLM, with a one-click copy of its suggestions so you can paste them into your file.

The tool lets you switch between Tabnine’s proprietary model, Mistral, GPT-3.5, or GPT-4.0 Turbo. For those that this feature is available to, Tabnine also offers a way to fine-tune the model using the customer’s codebase. As a generally applicable coding tool, it supports most popular languages, libraries, and IDEs. Tabnine can be deployed on-prem or on a virtual private cloud (VPC). The tool supports popular IDEs including VS Code, IntelliJ IDEs, Eclipse, and more.

Visual Studio IntelliCode

Visual Studio Intellicode provides context-aware autocompletion of whole code lines. It offers automated suggestions for common coding tasks and helps refactor your code by detecting repeated edits and suggesting the same edits elsewhere. The tool runs locally so you don’t have to worry about any code leaving your computer. Visual Studio IntelliCode is available in Visual Studio 2022 (C#, C++, Java, SQL, XAML) and in VS Code (TypeScript/JavaScript, Python and Java).

Code review tools

There are some AI tools for code review that can “peer review” your code to make sure it is solid before merging.

CodeRabbit

CodeRabbit is an open-source option that works in GitHub and GitLab to review your PRs. It offers line-by-line code review with one-click fixes to any problems discovered. You can just continue to create your PRs like you usually do. CodeRabbit will monitor the selected repositories to review all the new pull requests. The tool will also provide a summary of your MRs to make your code easier to understand e.g. for human reviewers.

CodeRabbit uses a variety of models including GPT-4 and GPT-3.5 Turbo, and claims to support all programming languages, although as usual, popular languages are better due to the availability of vast volumes of training data.

Codiga

Codiga is focused on code review via static code analysis. While it can produce code snippets, it’s designed to clean up your code. It can review and auto-fix issues, and you can set your own rules for code review. It provides security analysis to fix vulnerabilities right in your IDE (supported IDEs include the usual suspects: VS Code, IntelliJ IDEs, Visual Studio), and the tool also integrates with GitHub, GitLab, and Bitbucket. Codiga supports 15 different languages including JavaScript, TypeScript, C, C++, Python, PHP, Go, Java, Ruby, and Kotlin.

Standalone applications, CLI tools

There are some LLM-based code generation tools that work outside of your editor: either in a standalone application or (in the case of Cursor and Wing Python IDE Pro) offering a standalone code editor.

Aider

Aider is a CLI tool that lets you edit code in your local Git repository with LLMs: it provides a pair programming assistant in your terminal. Aider locally edits your files (it can edit multiple files at once), and will automatically Git commit changes for you. You’ll provide instructions for the LLM via natural language commands.

Supported languages include Python, JavaScript, TypeScript, PHP, HTML, CSS, etc. While Aider can connect to almost any LLM, its developers claim it works best with GPT-4o & Claude 3.5 Sonnet. One neat thing about Aider is that it lets you code with your voice!

AutoCodeRover

AutoCodeRover is an autonomous program improvement framework powered by LLMs. It’s a CLI tool that can automate software engineering tasks and is specialized on resolving GitHub issues (with debugging and feature additions via code generation). AutoCodeRover combines LLMs with analysis and debugging features to patch software code. It first analyzes the program structure to retrieve relevant code, then generates a patch. With a test suite available, the repair rate is even higher. AutoCodeRover performs very well: at the time of writing, it ranks 3rd on the SWE-bench leaderboard.

Cursor

Cursor is a standalone AI-first code editor based on VS Code that provides context-aware code completion, code generation, multi-line edits, and refactoring for your codebase. It uses GPT-4, GPT-4o, and Claude 3.5 Sonnet to predict edits that you can then just press tab on to use in your source code file. As usual, prompts may be added in natural language, and codebase answers (as well as codebase search) are also supported. Cursor can also access the web to query up-to-date information.

The editor is a fork of VS Code and claims to provide a familiar environment by importing extensions, themes, and keyboard shortcuts from that IDE. It offers a privacy mode to make sure your code is not stored by Cursor.

Devin

Devin is a fully autonomous AI software engineer that uses a proprietary model to write code, create websites, do unit testing, etc. It has its own command line for input prompts, a code editor, and a browser all in one place. Having a web browser is useful because Devin can not only learn from its own mistakes, but can also access other sites like the documentation of some technology you’re using in your code.

Devin claims to be able to tackle complex programming problems by first outlining a set of steps to take, then building the project using the same tools a human developer would use. It can generate, debug, and transpile code, and deploy the results (e.g. to a website). To try Devin, you’ll have to request access from Cognition AI.

OpenDevin

OpenDevin is an open-source alternative to Devin. It runs in a Docker container and can take care of complex software development tasks just like Devin. You can use OpenDevin with practically any LLM (including local LLMs through Ollama), but its documentation claims that the best models are GPT-4 and Claude 3.

OpenDevin currently ranks 15th on the SWE-bench Lite leaderboard, with its scores trailing those of Aider used with GPT-4o and Claude 3 Opus.

Replit AI

Replit AI provides personalized assistance based on your own codebase, with context-aware suggestions provided in response to natural language commands in a chat. It offers code completion, code generation, code editing, and the summarization of certain bits of code, all based on a model fine-tuned on your database.

Replit offers one-click copy and paste of the results with the “insert” feature to streamline the experience with the tool. It uses a proprietary model to offer the best answers with any of the 16 supported languages (Bash, C, C#, C++, CSS, Go, Java, JavaScript, HTML, PHP, Perl, Python, R, Ruby, Rust, and SQL - that said, Replit’s developer claims it works best with JavaScript and Python code). Replit also offers a mobile and a desktop app, making it a versatile option for developers who like to work on the move.

SWE-agent

Developed by researchers from Princeton University, SWE-agent can fix issues in GitHub repositories. It works by providing LM-centric prompts and feedback formats to make it easier for the model to inspect the repository and manage code files. SWE-agent can be run in a browser using GitHub codespaces in an in-browser VS Code environment, pulled from Docker, or installed for use as a CLI tool. Here’s a complete list of all the models it supports, with the most popular being GPT-4o, Claude Sonnet 3.5, DeepSeek Coder, etc. Ollama is also supported. At the time of writing this post, SWE-agent ranks 4th on the SWE-bench leaderboard.

Wing Python IDE Pro

Wing Python IDE Pro is another standalone code editor that was designed for, not surprisingly, Python. It can write and redesign code, and provides code inspection, refactoring, reformatting, and debugging features using OpenAI’s models. Wing Pro lets you work locally or on a remote host, VM, or container.

Online code generators

With some tools, you’ll use a web interface to interact with the LLM to generate your code. Here are some key LLM-based online code generators:

CodePal

CodePal is an online code generator that “speaks” a range of languages including C, C++, C#, Dart, Go, Java, JavaScript, Kotlin, PHP, Python, R, Ruby, Rust, Scala, TypeScript, and others. It provides the usual chat interface to interact with the LLM through natural-language prompts.

It can provide programming help by generating, testing, and debugging code, applying transformations and manipulations, as well as other coding tasks. CodePal provides a variety of helper services including a CI/CD writer, a Dockerfile writer, web design helpers, a mock data generator, a query writer, a code refactor tool, etc.

CodePal is built on top of GPT-3 and ChatGPT and offers a CLI tool and a Chrome extension.

Devika

Devika is another open-source alternative to Devin. It is an LLM-based software engineering tool that is able to interpret high-level human instructions, break them down into steps, research relevant information on the web, and write code to achieve your goals set in the prompt.

Devika supports Claude 3, GPT-4, Gemini, Mistral, Groq, and the use of local LLMs via Ollama. Using the Claude 3 family of LLMs is recommended. Devika has a web interface that lets you give instructions in the usual chat format.

Favtutor

While Favtutor primarily offers learning resources for coding, it also has a free online AI code generator that supports JavaScript, C++, Java, R, TypeScript, HTML5, C#, Python, Golang, Rust, PHP, and more (30+ programming languages in total). Based on OpenAI’s GPT-3.5 models, Favtutor’s tool can generate, debug, and convert code, and also offers data analysis functionality.

Documentation writing tools

Writing documentation is the perfect job for an LLM: the model has to interpret, explain, and document your code, something that LLMs were initially designed for. Documenting your code requires focus and manual work and as a recurring task, it’s an effort-intensive process. That makes documentation a low-hanging fruit for LLMs to help boost your software development productivity.

Mintlify Writer

Mintlify is available as an extension/plugin to VS Code and IntelliJ IDEs. The tool makes doc writing an effortless task: just highlight the code you want documented, hit your custom keyboard shortcut, and Mintlify will generate documentation for the selected code. The tool supports Python, JavaScript, TypeScript, C++, PHP, Java, C#, Ruby, Go & more. Docstring formats supported include JSDoc, reST, NumPy, DocBlock, Doxygen, Javadoc, GoDoc, XML, and Google. Mintlify also integrates with a range of analytics tools to help track viewer events, and offers a GPT-4 powered chatbot so users can get their questions answered by the LLM right away.

Mutable AI

Mutable AI promises to learn about your code to automatically document it, creating docs in a Wikipedia-style format. The tool can add citations linked to code, and diagrams to illustrate relationships in the repository. Users can “chat with the documentation” to get their specific questions answered, and the AI chat also offers a non-technical mode so that it can help the whole team. Naturally, the docs Mutable AI generates can be reviewed (either manually or using another LLM), and the tool will automatically keep your documentation up to date.

Other interesting AI-based applications

While not directly related to generating software code, there are some interesting LLM-based applications that we wanted to include since they are geared towards improving software development efficiency.

ClickUp

ClickUp is an all-in-one tool for brainstorming, project management, sprint planning, and software development. It can generate & review code and run unit tests, but the real feature here is the ability to define custom automations and even automated spring management. ClickUp has 1000+ integrations including Jira, Google Calendar, Slack, GitHub, GitLab, Loom, Bitbucket, etc. and also offers a Chrome extension.

Snyk Code

Snyk Code provides real-time static application security testing (SAST) powered by DeepCode AI. The tool identifies security vulnerabilities, helps prioritize top risks, and provides suggestions for fixing these code issues. Snyk Code supports 10 of the most popular programming languages including C#, C++, Go, Java, JavaScript, PHP, Python, Ruby, and TypeScript. You can use it in the CLI, or through one of its plugins and extensions for popular IDEs (Eclipse, VS Code, Visual Studio, IntelliJ IDEs).

Low-code and no-code tools

Finally, we thought it would be fun to list some of the no-code and low-code tools that you can check out to see if AI really is about to steal your job 👀. These make programming more accessible for non-developers, essentially “democratizing” software development.

AI2sql

AI2sql provides an AI-backed way to generate efficient and accurate SQL queries. It also supports schema visualization and syntax checking, and can explain and optimize queries.

Bubble

Bubble is a standalone no-code web application development platform. It lets you design, develop, test, launch applications using LLMs. It offers features like a drag-and-drop editor, pre-built components, an AI page generator, as well as built-in database and logic capabilities.

Debuild

Debuild is a low-code development tool that is used to accelerate and simplify the development of web applications. It generates React components, SQL code, and also helps users put together a visual interface.

Seek

Seek provides generative AI for data queries, like AI2sql. The tool converts natural-language queries into SQL code, making information from large databases more accessible.

Summary: LLM tools for software development

So these are some of the most popular LLM-based tools for supporting software development work. Whether it’s about code generation, writing tests, or generating documentation, the above tools have the potential to make you more productive. Make sure you check out each provider’s policies to get a clear picture of what exactly happens to your code if you decide to use any of the above tools.

Did we miss your favorite AI tool or overlook a feature of an existing tool? Drop us a line