Langtail Roadmap, Feedback and Changelog

Lot of open source models like LLama3, Phi, Mixtral are getting better and better. Langtail should support testing & deploying these as well

New assertion type that checks if the LLM output matches given requirements, using a language model to grade the output based on the rubric

Develop a Node.js client for Langtail, allowing developers to easily integrate and interact with Langtail services in their Node.js applications.

Currently there is no way to force a function/tool call using the langtail UI via the tool_choice parameter

Be able to override the configuration in prompt API - like change the used model version, max_tokens, etc

Sometimes, I am not able to reproduce the output I see on Langtail elsewhere even after retrying multiple times. It'll be great if I could see the raw request and response from LLM provider on each generation in the chat UI. This is also helpful to know other response values like system fingerprint while developing a prompt

Langtail highlights Markdown elements in LLM's answer. Let's add this to prompts. It would be especially for long (50k+) prompts.

Integrate GPT Vision capabilities into the Playground, allowing users to experiment and explore the vision-related features of GPT models.

Enhance the Playground to enable the execution of parallel functions, allowing developers to run multiple functions concurrently for improved efficiency and performance.

Add the functionality to directly open logs in the Playground for easier debugging and analysis.