Intro to chatting with LLMs

Using chatlas

chatlas: an opinionated LLM framework


  • One interface to many LLMs
  • Streaming output
  • Tool calling, MCP, etc
  • Structured data extraction
  • Async/non-blocking
  • Evals, OTel, etc
  • And more…

There are many frameworks – why chatlas?

  • I’m the author :)
  • Backed by Posit – a company with 15+ years of heavy investment in open source
  • Open source, MIT license
  • Simpler to navigate than “mega-frameworks” (e.g., LangChain, LlamaIndex, etc.)
  • Similar in spirit to Pydantic AI and llm, but different focus:
    • Interactive streaming “just works”
    • Delightful integration with shinychat

Many providers, one interface

import chatlas as ctl

chat1 = ctl.ChatAnthropic()
chat2 = ctl.ChatBedrockAnthropic()
chat3 = ctl.ChatGoogle()

Dozens of Chat{$Provider} available.

Many providers, one interface

import chatlas as ctl

chat = ctl.ChatBedrockAnthropic()

chat.chat("Teach me Python")

Switch providers without changing downstream code.

Model hints

See what model IDs are available for each provider.

💡 Expressed as a single string: ChatAuto("openai/gpt-5.5").

💡 If no model is provided, a sensible default is provided.

Streaming “just works”

Multi-turn: the chat remembers

import chatlas as ctl
chat = ctl.ChatBedrockAnthropic()
chat.chat("My name is Carson.")
chat.chat("What's my name?")

Your name is Carson.

LLMs are inherently stateless, but chat objects are stateful — they accumulate conversation history.

Working with chat state

Chat objects are stateful — they accumulate conversation history.

chat                      # see the history
chat.get_turns()          # obtain list of user/assistant turns
chat.set_turns([])        # reset to start fresh

Multi-modal input

Submit input other than text, such as images, pdfs, and more.

chat.chat(
  ctl.content_image_url("https://www.python.org/static/img/python-logo.png"),
  "Can you explain this logo?"
)

The Python logo features two intertwined snakes in yellow and blue, representing the Python programming language. The design symbolizes…

System prompt

Behind-the-scenes information that shapes every response — invisible to the user.

chat = ctl.ChatBedrockAnthropic(
    system_prompt="You are Jerry Seinfeld."
)

chat.chat("Tell me a joke")

What is the deal with passwords? You need a capital letter, a number, a symbol… at this point I need a degree just to log into my email.

System prompt tips

  1. Set the scene (e.g., “You are a dashboard chat assistant”).
  2. Define (un)desirable behavior (e.g., “Be concise”).
    • Give examples
  3. Give missing context (e.g., “Today’s date: 2026-07-13”).
    • Simple way to “inject” knowledge into the model without RAG or tool calling.

Chat methods

Three methods for interactive use:

  • .chat() — “echo” output, returns final response
  • .console() — open dedicated Python console
  • .app() — open dedicated web app

Two methods for programmatic use:

  • .stream() — returns a generator, you control output
  • .stream_async() — async version of .stream()

Streaming, in a nutshell

Similar to .chat(), but you control where each chunk of output goes.

import chatlas as ctl

chat = ctl.ChatBedrockAnthropic()

stream = chat.stream("My name is Carson.")
for chunk in stream:
    print(chunk)

Hello
Chatlas.
How
can
I
help?

Exercise

  1. Open up exercises/00-chat-hello.py.
  2. Add a custom system prompt to change the model’s behavior.
  3. Try using .app() – how is this similar/different from ChatGPT, etc?
  4. Quit the app, and programmatically determine the number of turns in the conversation history.
  5. Go to the API reference – find the Chat method for computing cost.