Category Archives: New (to me)

New Gee Whiz stuff I find

Cleaning Up Danglers on Gitorious

We’ve had problems lately with the server running out of space. It seems to hearken back to two items:

  1. We didn’t allocate enough space on the VM in the first place
  2. The external drive to which the backups are supposed to be stored seems to go down alot. So the backup is staying on the local box. Since it’s about 5 GB, the space fills up fast.

We finally allocated more space for the hard drive. But, in the interim, some of the gitorious files seemed to have been corrupted on creation. Not everyone was having problems, which made it difficult to diagnose. The first item that showed up was this:

ActionView::Template::Error (/var/www/gitorious/repositories/raisr/raisr_detailselectronics.git):
    24: 
    25:
<div class="gts-events">

26: <% prev_date = nil %> 
27: <% events.map { |event| EventPresenter.build(event, self) }.each do |event| %> 
28: <% if prev_date != event.created_at.strftime("%A, %B %e %Y") %> 
29:
<p class="gts-date-heading">30: <span class="gts-date-heading-inner"> app/models/repository.rb:250:in `new' 
  app/models/repository.rb:250:in `git' 
  app/presenters/event_presenter/push_summary_event.rb:165:in `fetch_first_commit' 
  app/presenters/event_presenter/push_summary_event.rb:132:in `initialize_commits' 
  app/presenters/event_presenter/push_summary_event.rb:31:in `initialize' 
  app/presenters/event_presenter.rb:28:in `new' 
  app/presenters/event_presenter.rb:28:in `build' 
  app/views/events/_events.html.erb:27:in `block in _app_views_events__events_html_erb___4043706570647562128_49011240' 
  app/views/events/_events.html.erb:27:in `map' app/views/events/_events.html.erb:27:in `_app_views_events__events_html_erb___4043706570647562128_49011240' 
  app/views/site/dashboard/_activities.html.erb:23:in `block in _app_views_site_dashboard__activities_html_erb__1312100781819782992_61802240' 
  app/helpers/tabs_helper.rb:89:in `block in tab_content' 
  app/helpers/tabs_helper.rb:88:in `tab_content' 
  app/helpers/tabs_helper.rb:55:in `block in tabbable' app/helpers/tabs_helper.rb:44:in `tabbable' app/helpers/tabs_helper.rb:61:in `pjax_tabbable' 
  app/helpers/tabs_helper.rb:35:in `activity_tabbable' 
  app/views/site/dashboard/_activities.html.erb:22:in `_app_views_site_dashboard__activities_html_erb__1312100781819782992_61802240' app/views/site/dashboard.html.erb:24:in `_app_views_site_dashboard_html_erb__754282507050535463_62777760' 
  app/controllers/site_controller.rb:123:in `block (3 levels) in render_dashboard' 
  app/controllers/site_controller.rb:111:in `block in render_dashboard' 
  app/controllers/application_controller.rb:339:in `paginate' 
  app/controllers/site_controller.rb:110:in `render_dashboard' 
  app/controllers/site_controller.rb:146:in `render_global_index' 
  app/controllers/site_controller.rb:39:in `index'

I ended up fixing that one by deleting the entry for that particular repository from the repositories table in the database, and by removing the one related comment that appeared in the committership table.

Before I did this, no one could get onto the Gitorious web interface. Afterwards, a few people were having problems, which made it a bit more difficult to debug. But then I saw this in the production log:

NoMethodError (undefined method `list_as_favorite?' for nil:NilClass):
 app/models/dashboard.rb:39:in `block in favorites'
 app/models/dashboard.rb:39:in `select'
 app/models/dashboard.rb:39:in `favorites'
 app/presenters/dashboard_presenter.rb:51:in `favorites'
 app/controllers/site_controller.rb:121:in `block (3 levels) in render_dashboard'
 app/controllers/site_controller.rb:111:in `block in render_dashboard'
 app/controllers/application_controller.rb:339:in `paginate'
 app/controllers/site_controller.rb:110:in `render_dashboard'
 app/controllers/site_controller.rb:146:in `render_global_index'
 app/controllers/site_controller.rb:39:in `index'

A search of the web finally brought me to the following solution in the Gitorious forum that worked. Thank you to Marcin Kulik for the solution:

This looks to me like an issue with old data in the db.
You may try running the following command in the app directory, as "git"
user:
RAILS_ENV=production bundle exec rake fix_dangling_comments
fix_dangling_committerships fix_dangling_events fix_dangling_favorites
fix_dangling_memberships fix_dangling_projects fix_dangling_repositories
fix_missing_repos fix_missing_wiki_repos fix_system_comments

(this is a single line in case email formatting breaks it)

Basically these tasks will find and fix inconsistent data in the
database and hopefully solve your issue with dashboard.

Marcin

The solution produced the following output:

[fix_dangling_comments] removing 0 orphaned comments
[fix_dangling_comments] removing 0 orphaned comments
[fix_dangling_committerships] removing 0 orphaned committerships
[fix_dangling_committerships] removing 0 orphaned committerships
[fix_dangling_events] removing 0 events with missing target Project
[fix_dangling_events] removing 0 events with missing target User
[fix_dangling_events] removing 0 events with missing target Group
[fix_dangling_events] removing 0 events with missing target MergeRequest
[fix_dangling_events] removing 2 events with missing target Repository
Removing 1 dangling favorites for Repository
Removing 0 dangling favorites for MergeRequest
Removing 0 dangling favorites for Project
Removing 0 invalid memberships
[fix_dangling_projects] removing 0 orphaned projects
[fix_dangling_projects] removing 0 orphaned projects
[fix_dangling_repositories] removing 0 orphaned repositories
[fix_dangling_repositories] removing 0 orphaned repositories
[fix_dangling_repositories] removing 0 orphaned repositories
[fix_dangling_repositories] removing 0 orphaned repositories
Attempting to fix 0 projects without wiki repo
Disabling edition of 0/0 merge request version comments

Authentication for Knockout/C# Web Application

We have an interesting situation: we’re developing a web application, using MVC4 that is heaving in Knockout. Mostly, I think, because the development thing thought it was cool, but also because the primary purpose of this application was to present data to the users from a database. That data needs to be presented in a number of different combinations. With KnockoutJS’s reputation for making tasks like that easier, it seemed to be a good fit.

However, not only will the data need to be presented on the web, it needs to expose web services so that users can use the data within their own applications.

And, on top of all of this, the user needs to be authenticated, and their level of permission known. This so we can determine which set of data to present (there will be multiple ones) and whether or not the user can do CRUD operations on the data.

So, I was presented the task of implementing user authentication. Since it took me a bit of time to figure it out, and I didn’t find alot of help on the web, I wanted to present what I did in order to help some other unlucky soul.

I should mention that this is code first programming. And that only the pertinent methods are shown in the Controllers; the CRUD you can figure out yourself.

Entities

    public class NameBaseEntity : IEntity
    {
        [Required]
        [DatabaseGeneratedAttribute(
              DatabaseGeneratedOption.Identity),Key()]
        public int Id { get; set; }

        [Required]
        [StringLength(256)]
        public string Name { get; set; }

        public DateTime Created_On { get; set; }
        public DateTime Modified_On { get; set; }
    }
    [Table("Group")]
    public class Group: NameBaseEntity
    {

    }
    [Table("User")]
    public class User: NameBaseEntity
    {
        [Required]
        public String Password { get; set; }
    }
   [Table("XRefUserGroup")]
    public class XRefUserGroup: IEntity
    {
        [Required]
        public int UserId { get; set; }

        [Required]
        public int GroupId { get; set; }

        [Key]
        public int Id
        {
            get;
            set; 
        }
        [Required]
        public System.DateTime Created_On
        {
            get;
            set;
        }
        [Required]
        public System.DateTime Modified_On
        {
            get;
            set;
        }
    }

Concrete Classes

    public class UserAuthentication: EntityOperations
    {

        public UserAuthentication(EFDbContext context, DbSet dataSet)
            : base(context, dataSet)
        {}

        /// Determine whether this user has provided the correct username and
        /// password to allow access to the system. If the user is valid,
        /// return the user id. If not, return 0.
        public int AuthenticateUser(String username, String password)
        {
            try
            {
                var userId = (from user in this._context.User
                              where user.Name == username && user.Password == password
                              select user.Id).Single();

                return userId;
            }
            catch (InvalidOperationException)
            {
                return 0;
            }

        }
    }
    public class UserAuthorization : EntityOperations
    {

        public UserAuthorization(EFDbContext context, DbSet dataSet)
            : base(context, dataSet)
        { }

        public bool IsUserMember(int UserId, int GroupId)
        {
            var isMember = from entry in this._context.XRefUserGroup
                           where entry.UserId == UserId && entry.GroupId == GroupId
                           select entry;
            return (isMember.Any());
        }

        public Dictionary<String, String> IsMemberOf(int UserId)
        {
            Dictionary<String, String> returnList = new Dictionary<String, String>();
            var memberGroups = from xref in this._context.XRefUserGroup
                               join groupName in this._context.Group on xref.GroupId equals groupName.Id
                               where xref.UserId == UserId
                               select new { GroupId = xref.GroupId, GroupName = groupName.Name };

            try
            {
                foreach (var dict in memberGroups)
                {
                    returnList.Add(dict.GroupId.ToString(), dict.GroupName);
                }
            }
            catch (InvalidOperationException) { }
            return returnList;                   
        }
    }

Controllers

   public class GroupController : Controller
    {
        private EFDbContext db = new EFDbContext();

       //
       // POST: /Group/PostGroupMembership/5
        public ActionResult PostGroupMembership(int memberId)
        {

            DbSet dbGroup = db.XRefUserGroup;

            HomeController controller = new HomeController();

            UserAuthorization auth = new UserAuthorization(db, dbGroup);
            UserAuthentication ua = new UserAuthentication(db, db.User);

            Dictionary<String, String> groupMembership = auth.IsMemberOf(memberId);

            User user = ua.GetItemById(memberId);

            Session.Add("Id", memberId);
            Session.Add("Uname", user.Name);
            Session.Add("Groups", String.Join(", ", groupMembership.Select(m => m.Key + ":" + m.Value).ToArray()));
            try
            {
                var tmp = groupMembership.Values.Where(p => p == "Administrator");
                Session.Add("IsAdmin",true);

            }
            catch (InvalidOperationException)
            {
                Session.Add("IsAdmin", false);
            }
            return RedirectToAction("Index", "Home");
        }
    public class UserController : ApiController
    {
        private DbContext db = new EFDbContext();

        /// Determine whether the user is authorized to use this system.

        public int GetAuthenticateUser(String user, String pass)
        {
            DbSet dbUser = db.User;

            UserAuthentication ua = new UserAuthentication(db, dbUser);

            return ua.AuthenticateUser(user, pass);
        }

        [HttpPost]
        public int PostAuthenticateUser(JObject identification)
        {
            DbSet dbUser = db.User;

            IList keys = identification.Properties().Select(p => p.Name).ToList();

            var obj = JObject.Parse(keys[0]);

            UserAuthentication ua = new UserAuthentication(db, dbUser);

            return ua.AuthenticateUser(obj["username"].Value(),obj["password"].Value());
        }
   }

KnockoutJS Code on the HTML Pages

var User = function (username, password) {
    var self = this;
    self.username = ko.observable();
    self.password = ko.observable();
    self.id = ko.observable(0);
    self.isAdmin = ko.observable(false);
    self.membership = ko.observable();
    self.hasError = ko.observable(false);

    self.id.subscribe(function (newValue) {
        this.hasError(newValue && newValue == 0);
    }, this);

    self.showName = function () {
        alert(self.username());
    };

    self.postLogin = function () {
        var jsonData = ko.toJSON(this);
        $.ajax({
            url: "/api/User/PostAuthenticateUser",
            type: "POST",
            data: jsonData,
            success: function (data) {
                self.id(data);
                if (data > 0) {
                    window.location.replace("/Group/PostGroupMembership?memberId=" + data);

                } else {

                    self.hasError(true);
                }
            },
            error: function () {
                self.id = 0;
            }
        });
    };

    return this;
};

On the Index page

    
   var Member = function () {
   var self = this;
   self.username = ko.observable('@Session["Uname"]');
   self.id = ko.observable('@Session["Id"]');
   self.isAdmin = ko.observable('@Session["IsAdmin"]' == 'True');
   self.membership = ko.observable('@Session["Groups"]');
};
$(document).ready(function () {
    var test = new Member();
    ko.applyBindings(test);
});

Python SUDS Library Client

I was trying to create a ‘quick and dirty’ python SOAP client and was finding it difficult. The WSDL I was dealing with wasn’t a simple one, which, of course, made creating a client a bit more difficult. Add to the fact that I’m far from a python expert (have been tired of learning the ‘latest/greatest’ new toy), so finally started learning python reluctantly.

I found this post in StackOverflow that listed a number of python SOAP libraries and I started working through the examples. I finally settled on using SUDS. It’s not the most up-to-date library – the last development done on in was in 2010. But frankly, it’s the only one from which I could obtain results.

That being said, it wasn’t as straightforward as I would have liked. My first language is PERL; my second is Java. So maybe ‘thinking in python’ was not as easy for me. But still, I have found it a good language to work with, which is why I have been forcing myself to use it instead of PERL lately.

So, my code…..

There are groups of parameters in the WSDL I’m using, which required me to create dictionaries of dictionaries:

sort = dict(sortPolicyId='RELEVANCE',order='DESCENDING')
criteria = dict(searchPhrase='test', sortPolicy=sort)
params = dict(maximumResults=1000, searchTimeOut=24000)
contextParams = dict(ContextualCriteria=criteria,SearchCriteriaParameters=params)

The next part, passing them, does come directly from the documentation:

results = client.service.contextualSearch(**contextParams)

Then the next hard part: I couldn’t figure out how to get the values out. What I finally found was an example for pysimplesoap that I could use: Pull the results from the results. That is:

resultList = results.results

At this point, I had something from which I could pull values:

for item in resultList:
    print item.catalogEntry.id

… which gave me the values I needed.