This took me some time. The problem was occurring only on https, not on http. So ruby code, gems, and versions seemed to be fine.

Turns out I needed this line in my apache config:

RequestHeader set X_FORWARDED_PROTO 'https' env=HTTPS

Actually, depending on the apache version, this probably works better:

  RequestHeader set "X-Forwarded-Proto" expr=%{REQUEST_SCHEME}
  RequestHeader set "X-Forwarded-SSL" expr=%{HTTPS}

And for nginx, it is something similar:

proxy_set_header X-Forwarded-Proto https;