I just updated my, and my R.R. teams, website Trendercast. I made it far more responsive with:

  1. an OAuth sign-in process that won’t navigate away from your page (wont interrupt podcast playing)
  2. a submission notification whenever new RSS feeds are entered (they take a while to gather and process)
  3. Threaded RSS feed updates!!!

This is my first time trying out threads and I’m very happy with it.

I experimented in IRB like a good developer and found that calling the join method on a thread will have the “program” wait for the thread to finish before resuming. This is one feature I wanted to use in Rails … but guess what… using thread.join in Rails gridlocks the application. Thankfully I found a way around that. Here’s basically what the code looks like.

module FeedGenerator
  def self.gen_feed(url, view_env)
    # fetch record for RSS feed
    feed = Feed.where(url: url).take(1).first
    # If the feed is older than 4 hours we check to see if a new podcast has been posted
    # but we don't want to delay the user's page loading so lets just start a thread in the
    # background to perform fetching and DB updating
    threads[url] = { perform(url) } unless updated?(feed)

    # if no record exists
    if feed.nil?
      # have the current process wait until data is finished being retrieved
      sleep 3 while threads[url].alive?

      # Allow the garbage collector to remove dead threads
      threads[url] = nil 

      # We have to change the query so Rails won't use the previously cached result
      # Grab the newly made feed
      feed = Feed.where(url: url).where("created_at > ?", - 20.minutes).take(1).first
    end, view_env)

  def self.threads
    @threads ||= {}

  def self.perform(url)
    # grab RSS feed data and save to DB

  def self.updated?(feed)
    if feed
      if feed.updated_at.between?(,
        return true

The server runs great and the experience is everything I had hoped it would be. My next step is to write a sleeper agent in JavaScript when there is an update to be checked for and dynamically insert it in the page once it’s available.

1 Like

I might need to consider other users starting a thread for the same url at the same time. Perhaps I should rewrite the one line to:

# if feed is not updated and thread isn't alive (or exists)
if !updated?(feed) and !threads[url].try{|t| !t.alive? }
  threads[url] = { perform(url) }

There are things I don’t know… like if there are separate Ruby processes for every user or not. That would change what I would need to look for in threading.

1 Like

This seems like a recent up-to-date article, any help to you?


Thanks Aston! It’s very informative. It doesn’t cover my curiosity in how Rails works. I have a general idea about Rack have a per route listener (thread)… but I’ve only imagined about how it might work.

There are many things I wonder about. Like in Minitest I would want to do things in a test file with just describe and it blocks. In Rails I’ve found tests to be very strict about what you can do based on what folder you’re in and a basic need for inheritance to add abilities.

There are “some” people out there who know these things and if they happen to come by and speak up I’m most delighted to learn. Otherwise I’d need to invest a lot of time studying source code. I’m not against that, I’ll get around to that when I need to. I’d rather have connections and dialog than be a loner who’s read the source to the whole world.

1 Like

Yeah I totally hear you… hopefully when we get the new look a few new people will look in :smiley:

Not sure if you are keeping a eye on the Elixir Forum but José Valim kindly created a section on the official Elixir lang site for us! I’m hoping the powers that be in Ruby world might do the same :003:

Here’s hoping anyway :lol:

(Sorry to go off-topic - please feel free to delete or move to a separate thread)

1 Like

If I understand you, some of those answers depend on how you deploy rails. Unicorn vs. puma vs. whatever.

1 Like

@kofno Yeah, I do know Puma is thread oriented and Heroku is promoting it over Unicorn now.

On the topic of threads this showed up on my Twitter feed: Ruby Threads and ActiveRecord Connections and it was written a week ago. Good insight on database connection/pooling with threading.

And from what I hear Celluloid is the best system for handling threads in Ruby (even better in JRuby) and Sidekiq is the best threaded background system which has become mature enough to graduate away from Celluloid and they’ve written it now with pure Ruby threads. That is pretty much the extent of my knowledge on threads.

That and Ruby’s “gil” is problematic for threads except with cases of IO.


@danielpclark you maybe also want to take a look on GitHub - ruby-concurrency/concurrent-ruby: Modern concurrency tools including agents, futures, promises, thread pools, supervisors, and more. Inspired by Erlang, Clojure, Scala, Go, Java, JavaScript, and classic concurrency patterns.
Sidekiq uses it now (In addition to bare bone threads) and also rails is using it. :slightly_smiling:

And I’m currently looking into it with a fun project: GitHub - tak1n/reifier: A rack app server written in pure ruby


If you want to learn more about ruby threads, I can recommend this book:


Celluloid use Fibers to organize his “tasks” flollowing the actor model into a single thread. But you can also use multiple thread each with there own pool of fibers (drived by Celluloid, of course). Be careful with JRuby : Fibers are real threads on this platform and you can reach the limit of your system. For MRI, it’s not a problem because fibers are not system threads (and should not be).


My threads are still working fine on the server. But they seem to have introduced an issue with DB access ActiveRecord::ConnectionTimeoutError. I need to somehow allow more access to the DB when threads start.

Spinning up a bunch of threads by checking all of the podcasts gets a bunch of threads reading for changes and writing updates. This can have some of the podcasts to seem to not load in the browser… probably the ones not in cache. :confused:

Have you introduced a connection pool?

Not yet. I’ve been thinking about it because I remember seeing blog posts about them.

I think I’ve got it!

  # if feed is not updated and thread isn't alive (or exists)
  if !updated?(feed) and !threads[url].try{|t| !t.alive? }
    threads[url] = { perform(url) }

def perform(url)
  ActiveRecord::Base.connection_pool.with_connection do
    # ...

I just wrapped the one place I was calling with the ActiveRecord::Base.connection_pool.with_connection and the site no longer comes to a halt. :smile:


:clap: :confetti_ball:

Glad you figured it out!


It turns out the Podcast feeds aren’t updating anymore. I think I might just write my own ActiveRecord thread pool with priority access.

1 Like

I don’t know why but it’s working without issue now for months.