Streaming Chat GPT with Rails + Hotwire

15 Jul, 2024

In the previous video, we set up a chat service to access ChatGPT and return responses. Now, we're taking it to the next level by enabling streaming responses. This way, as soon as ChatGPT starts responding, you’ll see the messages stream in live.

I've set up a basic UI with a form where you can post your questions. Currently, it doesn't keep any chat context or history - you just ask a question and get an answer.

Let’s dive into building it!

Setting Up Models

The first thing we'll need to do is generate some models to handle conversations and messages.

rails generate model Conversation user:references
rails generate model Message conversation:references role:text content:text

Now, let's run the migrations:

rails db:migrate

Defining Relationships

We'll need to set up associations between our models.

Conversation Model should look like this:

class Conversation < ApplicationRecord
  belongs_to :user
  has_many :messages, dependent: :destroy
end

And Message Model will have:

class Message < ApplicationRecord
  belongs_to :conversation
end

Creating Conversations

Next, we need to ensure that a conversation is created when a user visits the home page. We’ll do this in the PagesController.

pages_controller.rb:

class PagesController < ApplicationController
  skip_before_action :authenticate

  def home
    @conversation = Current.user.conversations.first_or_create!
  end
end

Also, in the User Model:

class User < ApplicationRecord
  # ...
  has_many :conversations, dependent: :destroy
  # ...
end

Adjusting the Form

We need to pass the conversation_id through the form. Here’s how we can set a hidden field in our form.

home.html.erb:

<%= form_with url: chats_path, local: true do |form| %>
  <%= hidden_field_tag :conversation_id, @conversation.id %>
  <%= form.text_field :content %>
  <%= form.submit "Post" %>
<% end %>

If we inspect the form, we should see the conversation ID included as a hidden field:

Handling Messages

Now, we'll update the chat controller to permit the conversation_id and create new messages.

chats_controller.rb:

class ChatsController < ApplicationController
  def create
    conversation = Current.user.conversations.find(chat_params[:conversation_id])
    message = conversation.messages.create!(
      role: "user",
      content: chat_params[:content]
    )
    ChatService.new(conversation: conversation, message: message).call

    head :no_content
  end

  private

  def chat_params
    params.permit(:content, :conversation_id).merge(user: Current.user)
  end
end

Chat Service Refinement

In our ChatService, we'll adjust it to handle messages from conversations.

chat_service.rb:

class ChatService
  attr_reader :message, :conversation

  def initialize(conversation:, message:)
    @conversation = conversation
    @message = message
  end

  def call
    messages = training_prompts.map do |prompt|
      { role: "system", content: prompt}
    end

    conversation.messages.each do |message|
      messages << { role: message.role, content: message.content }
    end

    response = client.chat(
      parameters: {
        model: "gpt-3.5-turbo",
        messages: messages,
        temperature: 0.3
      }
    )

    conversation.messages.create!(
      role: "assistant",
      content: response.dig("choices", 0, "message", "content")
    )

    true
  end

  # ...
end

Turbo Streams for Real-time Updates

We need to set up Turbo Streams to broadcast messages in real-time.

Update message.rb:

class Message < ApplicationRecord
  belongs_to :conversation

  after_create_commit -> { broadcast_created }
  after_update_commit -> { broadcast_updated }

  def broadcast_created
    broadcast_append_to(
      conversation,
      partial: "messages/message",
      locals: { message: self },
      target: "messages"
    )
  end

  def broadcast_updated
    broadcast_replace_to(
      conversation,
      partial: "messages/message",
      locals: { message: self },
      target: "message_#{id}"
    )
  end
end

And our message partial _message.html.erb:

<div id="message_<%= message.id %>" class="py-5 px-6 rounded-md mt-4 mr-10 bg-gray-100 text-gray-700">
  <%= message.content %>
</div>

Styling Messages

To differentiate between user and assistant messages, we’ll add some basic styles:

_message.html.erb:

<div id="message_<%= message.id %>" class="py-5 px-6 rounded-md mt-4 <%= message.role == "user" ? "ml-10 bg-blue-600 text-white" : "mr-10 bg-gray-100 text-gray-700" %>">
  <%= message.content %>
</div>

Streaming Setup

Finally, we'll set up streaming in our ChatService:

chat_service.rb (continued):

class ChatService
  attr_reader :message, :conversation

  def initialize(conversation:, message:)
    @conversation = conversation
    @message = message
  end

  def call
    messages = training_prompts.map do |prompt|
      { role: "system", content: prompt}
    end

    conversation.messages.each do |message|
      messages << { role: message.role, content: message.content }
    end

    new_message = conversation.messages.create!(
      role: "assistant",
      content: ""
    )

    response = client.chat(
      parameters: {
        model: "gpt-3.5-turbo",
        messages: messages,
        temperature: 0.3,
        stream: proc do |chunk, _bytesize|
          text = chunk.dig("choices", 0, "delta", "content")
          if text.present?
            new_message.content += text
            new_message.save
          end
        end
      }
    )
    true
  end
  # ...
end

Now, if we reload the page and ask a question, the responses will start streaming in!

Conclusion

With Turbo Streams and Rails, integrating a streaming chat feature using ChatGPT is straightforward. It feels snappy when texts stream in rather than waiting for the entire response.

I hope you found this tutorial useful! If you want to see more about integrating ChatGPT with your Rails app, let me know in the comments.

Stay tuned for more!

Discussion (0)

To comment you need to sign up for a free account or sign in.

Get new videos in your inbox weekly!

Be the first to hear about new courses and videos launching. You’ll only receive relevant news, no spam.