Tag Archives: corrupted repositories

Cleaning Up Danglers on Gitorious

We’ve had problems lately with the server running out of space. It seems to hearken back to two items:

  1. We didn’t allocate enough space on the VM in the first place
  2. The external drive to which the backups are supposed to be stored seems to go down alot. So the backup is staying on the local box. Since it’s about 5 GB, the space fills up fast.

We finally allocated more space for the hard drive. But, in the interim, some of the gitorious files seemed to have been corrupted on creation. Not everyone was having problems, which made it difficult to diagnose. The first item that showed up was this:

ActionView::Template::Error (/var/www/gitorious/repositories/raisr/raisr_detailselectronics.git):
    24: 
    25:
<div class="gts-events">

26: <% prev_date = nil %> 
27: <% events.map { |event| EventPresenter.build(event, self) }.each do |event| %> 
28: <% if prev_date != event.created_at.strftime("%A, %B %e %Y") %> 
29:
<p class="gts-date-heading">30: <span class="gts-date-heading-inner"> app/models/repository.rb:250:in `new' 
  app/models/repository.rb:250:in `git' 
  app/presenters/event_presenter/push_summary_event.rb:165:in `fetch_first_commit' 
  app/presenters/event_presenter/push_summary_event.rb:132:in `initialize_commits' 
  app/presenters/event_presenter/push_summary_event.rb:31:in `initialize' 
  app/presenters/event_presenter.rb:28:in `new' 
  app/presenters/event_presenter.rb:28:in `build' 
  app/views/events/_events.html.erb:27:in `block in _app_views_events__events_html_erb___4043706570647562128_49011240' 
  app/views/events/_events.html.erb:27:in `map' app/views/events/_events.html.erb:27:in `_app_views_events__events_html_erb___4043706570647562128_49011240' 
  app/views/site/dashboard/_activities.html.erb:23:in `block in _app_views_site_dashboard__activities_html_erb__1312100781819782992_61802240' 
  app/helpers/tabs_helper.rb:89:in `block in tab_content' 
  app/helpers/tabs_helper.rb:88:in `tab_content' 
  app/helpers/tabs_helper.rb:55:in `block in tabbable' app/helpers/tabs_helper.rb:44:in `tabbable' app/helpers/tabs_helper.rb:61:in `pjax_tabbable' 
  app/helpers/tabs_helper.rb:35:in `activity_tabbable' 
  app/views/site/dashboard/_activities.html.erb:22:in `_app_views_site_dashboard__activities_html_erb__1312100781819782992_61802240' app/views/site/dashboard.html.erb:24:in `_app_views_site_dashboard_html_erb__754282507050535463_62777760' 
  app/controllers/site_controller.rb:123:in `block (3 levels) in render_dashboard' 
  app/controllers/site_controller.rb:111:in `block in render_dashboard' 
  app/controllers/application_controller.rb:339:in `paginate' 
  app/controllers/site_controller.rb:110:in `render_dashboard' 
  app/controllers/site_controller.rb:146:in `render_global_index' 
  app/controllers/site_controller.rb:39:in `index'

I ended up fixing that one by deleting the entry for that particular repository from the repositories table in the database, and by removing the one related comment that appeared in the committership table.

Before I did this, no one could get onto the Gitorious web interface. Afterwards, a few people were having problems, which made it a bit more difficult to debug. But then I saw this in the production log:

NoMethodError (undefined method `list_as_favorite?' for nil:NilClass):
 app/models/dashboard.rb:39:in `block in favorites'
 app/models/dashboard.rb:39:in `select'
 app/models/dashboard.rb:39:in `favorites'
 app/presenters/dashboard_presenter.rb:51:in `favorites'
 app/controllers/site_controller.rb:121:in `block (3 levels) in render_dashboard'
 app/controllers/site_controller.rb:111:in `block in render_dashboard'
 app/controllers/application_controller.rb:339:in `paginate'
 app/controllers/site_controller.rb:110:in `render_dashboard'
 app/controllers/site_controller.rb:146:in `render_global_index'
 app/controllers/site_controller.rb:39:in `index'

A search of the web finally brought me to the following solution in the Gitorious forum that worked. Thank you to Marcin Kulik for the solution:

This looks to me like an issue with old data in the db.
You may try running the following command in the app directory, as "git"
user:
RAILS_ENV=production bundle exec rake fix_dangling_comments
fix_dangling_committerships fix_dangling_events fix_dangling_favorites
fix_dangling_memberships fix_dangling_projects fix_dangling_repositories
fix_missing_repos fix_missing_wiki_repos fix_system_comments

(this is a single line in case email formatting breaks it)

Basically these tasks will find and fix inconsistent data in the
database and hopefully solve your issue with dashboard.

Marcin

The solution produced the following output:

[fix_dangling_comments] removing 0 orphaned comments
[fix_dangling_comments] removing 0 orphaned comments
[fix_dangling_committerships] removing 0 orphaned committerships
[fix_dangling_committerships] removing 0 orphaned committerships
[fix_dangling_events] removing 0 events with missing target Project
[fix_dangling_events] removing 0 events with missing target User
[fix_dangling_events] removing 0 events with missing target Group
[fix_dangling_events] removing 0 events with missing target MergeRequest
[fix_dangling_events] removing 2 events with missing target Repository
Removing 1 dangling favorites for Repository
Removing 0 dangling favorites for MergeRequest
Removing 0 dangling favorites for Project
Removing 0 invalid memberships
[fix_dangling_projects] removing 0 orphaned projects
[fix_dangling_projects] removing 0 orphaned projects
[fix_dangling_repositories] removing 0 orphaned repositories
[fix_dangling_repositories] removing 0 orphaned repositories
[fix_dangling_repositories] removing 0 orphaned repositories
[fix_dangling_repositories] removing 0 orphaned repositories
Attempting to fix 0 projects without wiki repo
Disabling edition of 0/0 merge request version comments