Rewrite of primary gitolite interface for better resilience.

The primary "update_repositories" function is now largely self-correcting
against a variety of errors seen in the field.  Also added a bunch of
error checking on functions called-out to the shell.  Among other things,
this code attempts to reconnect the gitolite-admin public key when it
gets deleted (can happen).

Regular use of 'sys/fetch_changesets' will recorrect any errors in the
public key directory and gitolite.conf file. For example, orphan keys
that happen to be left behind in the public key directory will be removed,
missing keys will be added, etc.

One new piece of functionality is the 'recycle_bin' which is used to store
deleted repositories immediately after they are deleted (allowing them
to be recovered for up to 'gitRecycleExpireTime' hours after deletion).

You need to migrate plugins because there are new plugin settings:

    rake db:migrate_plugins

Author: kubitron

README.mkd

@@ -1,9 +1,9 @@
-# Redmine Git Hosting Plugin (v0.4.2)
+# Redmine Git Hosting Plugin (v0.4.2x)
 
 A ChiliProject / Redmine plugin which makes configuring your own git hosting easy. This plugin allows straightforward management
 of gitolite and associated public keys, the git daemon, and integrates code from Scott Schacon's "grack" utility
 to provide Git Smart HTTP access.  Git repositories are automatically created when the repository is created in
-redmine.  There is also an option to automatically create a git repository for a project, when the project is created.
+Redmine.  There is also an option to automatically create a git repository for a project, when the project is created.
 Caching functionality is also implemented to speed page-load times for viewing git repositories.
 
 
@@ -41,6 +41,9 @@ Note that this guide refers to the "web server user" as the user under which Rai
 not always) the same as the user that runs the main web server.  If you are running Rails under a different user, follow
 these instructions using that user, not the one for the main web server.
 
+As of the most recent set of patches, this plugin is compatible with running multiple Redmine installations on the same server, each
+with the same *or* different gitolite users/repositories.  The later configuration (multiple Redmine installations, each with a different
+gitolite installation) is particularly useful for a web-hosting scenario with independent developers.
 
 ## Step-By-Step configuration instructions
 
@@ -101,12 +104,12 @@ variables are set correctly.  To adjust these variables, open an editor and edit
 Starting on line 22 you will see the settings definitions you should edit.
 
 
-The *httpServer* variable should be set to the url used to access your redmine site, e.g. www.my-own-personal-git-host-server.com.  Note that if Redmine is not
-installed in the site root this should include the path to your redmine root, e.g. www.my-own-personal-git-host-server.com/path/to/redmine
+The *httpServer* variable should be set to the url used to access your Redmine site, e.g. www.my-own-personal-git-host-server.com.  Note that if Redmine is not
+installed in the site root this should include the path to your Redmine root, e.g. www.my-own-personal-git-host-server.com/path/to/redmine
 
 
 The *gitServer* will usually be the same as the the httpServer variable -- this is the server name to use to access the gitolite repositories via ssh.  This should be
-the hostname only, so this will be different from *httpServer* if Redmine is not installed in the site root.  In other words, even if redmine is installed in 
+the hostname only, so this will be different from *httpServer* if Redmine is not installed in the site root.  In other words, even if Redmine is installed in 
 www.my-own-personal-git-host-server.com/path/to/redmine, *gitServer* should be set to www.my-own-personal-git-host-server.com
 
 
@@ -146,12 +149,51 @@ you have both git and svn (or hg, or cvs etc.) repositories, this may cause prob
 
 *Delete Git Repository When Project Is Deleted* can be enabled to let this plugin control repository deletion as well as repository creation.  By default, 
 this feature is disabled and when a repository is deleted in ChiliProject / Redmine, it is not deleted in gitolite.  This is a safety feature to prevent 
-the accidental loss of data. If this feature is enabled, the safety is turned off and the repository files will be permanently deleted when the Project/Repository is deleted in ChiliProject/Redmine.
+the accidental loss of data. If this feature is enabled, the safety is turned off and the repository files will be deleted when the Project/Repository is 
+deleted in ChiliProject/Redmine.  Note, however, that even when this feature is enabled, deleted repositories are placed into a "recycle_bin" for a configurable
+amount of time (defaulting to 24 hours) and can be recovered by recreating the project in Redmine with the same Identifier. Details are placed in the log.
+
+The *gitRecycleBasePath* is the path *relative to the git user root* where deleted repositories are placed.  This path should end in a path separator, e.g. '/'.  
+Deleted repositories are kept here for up to *gitRecycleExpireTime* hours (configurable, defaults to 24.0 hours).
+
+The *gitLockWaitTime* represents the amount of time that the plugin will wait in attempting to acquire its internal synchronization lock before giving up.  
+You probably will not need to change this value. 
 
 *Show Checkout URLs* can be disabled to hide the git URL bar in the repository tab.  It is enabled by default.
 
 See below in the "Caching" section of this readme for more information on caching and how the caching variables should be configured.
 
+## Resychronization of gitolite configuration 
+
+Whenever a Redmine `fetch_changesets()` operation is executed (i.e. `http://REDMINE_ROOT/sys/fetch_changesets?key=xxx`), this plugin will check the
+gitolite keydir and configuration file for consistency.  It will correct any errors that it finds.  Further, regular execution of a fetch_changesets operation
+will make sure that repositories placed in the recycle_bin (during delete operations) will be expired and removed.  Since there still seem to be some 
+phantom synchronization problems, it is recommended that you execute `fetch_changesets()` regularly (every 15 to 30 minutes).
+
+Two rake tasks can additionally be used for resynchronization (although these are redundant with executing `fetch_changesets()` through other means).  
+
+**(1)** To fixup the gitolite configuration file, fix errors, and delete expired files in the recycle_bin, execute:
+
+    RAILS_ENV=production rake gitolite:update_repositories
+
+**(2)** To perform all the above operations while at the same time fetching changesets for all repositories, execute:
+
+    RAILS_ENV=production rake gitolite:fetch_changes
+
+**Note that it is very important that these commands be run as *www-user* (or whatever your web server user happens to be), lest you get permission problems later.** 
+(The same is true of any `fetch_changesets()` operation initiated without using the web server, i.e. through the command line or from the cron daemon).
+
+## Interaction with non-Redmine gitolite users
+
+This plugin respects gitolite repositories that are managed outside of Redmine or managed by both Redmine and non-Redmine users:
+
+* When performing a *fetch_changesets()* operation, this plugin will delete and reestablish all keys that are of the form "redmine_", 
+  since it considers these to be under its exclusive control.  A special token, called "redmine_dummy_key", is used as a placeholder when no access 
+  is granted for a given repository.
+* Keys other than "redmine_" are left untouched and can be in projects by themselves or mixed in with projects managed by redmine.
+* When a Redmine-managed project is deleted (with the *Delete Git Repository When Project Is Deleted* option enabled), its corresponding git repository 
+  *will not be deleted/recycled* if there are non-Redmine keys in the gitolite.conf file.
+
 ## A Note About PATH variables
 
 One major source of issues with this plugin is that Rails needs to be able to run both *sudo* and *git*.  Specifically, these programs need to be in one of the directories specified by
@@ -246,27 +288,27 @@ This library allows you to quickly deploy ChiliProject, with this plugin to an u
 chili\_test.sh script, modifying the variables in those scripts as desired.  This library is still under development,
 so these instructions may need to be updated in the near future.
 
-## Selinux Configuration for redmine
+## Selinux Configuration for Redmine
 
 This plugin can be configured to run with selinux.  We have included a rakefile in tasks/selinux.rake to assist
 with installing with selinux.  You should start by editing init.rb and migrating as described above.  Then, you 
-can execute one of the selinux rake tasks (from the redmine root).  For instance, the simplest option installs 
-a selinux configuration for both redmine and the redmine_git_hosting plugin:   
+can execute one of the selinux rake tasks (from the Redmine root).  For instance, the simplest option installs 
+a selinux configuration for both Redmine and the redmine_git_hosting plugin:   
 
     rake selinux:install RAILS_ENV=production 
 
 This will generate the redmine_git_hosting binaries in ./bin, install a selinux policy for these binaries (called
-redmine_git.pp), then install a complete context for redmine as follows:
+redmine_git.pp), then install a complete context for Redmine as follows:
 
-**(1)** Most of redmine will be marked with "public_content_rw_t". 
+**(1)** Most of Redmine will be marked with "public_content_rw_t". 
 
 **(2)** The dispatch files in Rails.root/public/dispatch.* will be marked with "httpd_sys_script_exec_t"
 
 **(3)** The redmine_git_hosting binaries in Rails.root/vendor/plugins/redmine_git_hosting/bin will be labeled
 with "httpd_redmine_git_script_exec_t", which has been crafted to allow the sudo behavior required by these
 binaries.
 
-Note that this rake file has additional options.  For instance, you can specify redmine roots with regular 
+Note that this rake file has additional options.  For instance, you can specify Redmine roots with regular 
 expressions (not globbed expessions!) as follows (notice the use of double quotes):
 
     rake selinux:install RAILS_ENV=production ROOT_PATTERN="/source/.*/redmine"
@@ -288,7 +330,8 @@ with apache and fcgi. Other configurations may require slight tweaking.
 
 This plugin has been primarily tested on Ubuntu Server 10.10 and 11.04 (32 and 64 bit) with ChiliProject v1.x,
 ChiliProject 2.0.0 and Redmine 1.2.1 with PostgreSQL as the database (July, 2011).  It is possible that some
-debugging will be necessary for other configurations.
+debugging will be necessary for other configurations.  Selinux configurations were tested under Redhat Enterprise Linux 
+version 6.x with apache and fcgi.
 
 
 ## Required gems
@@ -308,7 +351,8 @@ This plugin is based largely on the Gitosis plugin by Jan Schulz-Hofen for http:
 were provided by github users untoldwind, tingar and ericpaulbishop. These updates were merged together and
 expanded upon by Eric Bishop to create this more comprehensive Git Hosting plugin.
 
+Copyright (c) 2011 John Kubiatowicz (kubitron@cs.berkeley.edu) MIT License.
+
 Copyright (c) 2010-2011 Eric Bishop (ericpaulbishop@gmail.com) MIT License.
 
 Copyright (c) 2009-2010 Jan Schulz-Hofen, ROCKET RENTALS GmbH (http://www.rocket-rentals.de). MIT License.
-

app/models/git_hosting_observer.rb

@@ -3,6 +3,7 @@ class GitHostingObserver < ActiveRecord::Observer
 
 	@@updating_active = true
 	@@updating_active_stack = 0
+	@@updating_active_flags = {}
 	@@cached_project_updates = []
 
 	def reload_this_observer
@@ -13,6 +14,12 @@ def reload_this_observer
 
 
 	def self.set_update_active(is_active)
+		case is_active
+                	when Symbol then @@updating_active_flags[is_active] = true
+                	when Hash then @updating_active_flags.merge(is_active)
+                	when Project then @@cached_project_updates << is_active
+                end
+
 		if !is_active
                 	@@updating_active_stack += 1
                 else
@@ -23,22 +30,22 @@ def self.set_update_active(is_active)
                 end
 
 		if is_active && @@updating_active_stack == 0
-			if @@cached_project_updates.length > 0
+			if @@cached_project_updates.length > 0 || !@@updating_active_flags.empty?
 				@@cached_project_updates = @@cached_project_updates.flatten.uniq.compact
-				GitHosting::update_repositories(@@cached_project_updates, false)
+				GitHosting::update_repositories(@@cached_project_updates, @@updating_active_flags)
                         	@@cached_project_updates = []
+                        	@@updating_active_flags = {}
 			end
                 	@@updating_active = true
                 else
                 	@@updating_active = false
 		end
 	end
 
-
 	def before_destroy(object)
 		if object.is_a?(Repository::Git)
 			if Setting.plugin_redmine_git_hosting['deleteGitRepositories'] == "true"
-				GitHosting::update_repositories(object.project, true)
+				GitHosting::delete_repository(object.project)
 				%x[#{GitHosting::git_user_runner} 'rm -rf #{object.url}' ]
 			end
 			GitHosting::clear_cache_for_project(object.project)
@@ -63,7 +70,7 @@ def before_save(object)
 	def after_save(object)
 		update_repositories(object)
 	end
-
+ 
 
 	def after_destroy(object)
 		if !object.is_a?(Repository::Git)
@@ -74,9 +81,7 @@ def after_destroy(object)
 
 	protected
 
-
 	def update_repositories(object)
-
 		projects = []
 		case object
 			when Repository::Git then projects.push(object.project)
@@ -87,7 +92,7 @@ def update_repositories(object)
 		end
 		if(projects.length > 0)
 			if (@@updating_active)
-				GitHosting::update_repositories(projects, false)
+				GitHosting::update_repositories(projects)
 			else
 				@@cached_project_updates.concat(projects)
 			end

app/models/git_hosting_settings_observer.rb

@@ -1,74 +1,112 @@
 class GitHostingSettingsObserver < ActiveRecord::Observer
 	observe :setting
 
-	@@old_hook_debug   = Setting.plugin_redmine_git_hosting['gitHooksDebug']
-	@@old_hook_asynch  = Setting.plugin_redmine_git_hosting['gitHooksAreAsynchronous']
-	@@old_http_server  = Setting.plugin_redmine_git_hosting['httpServer']
-	@@old_git_user     = Setting.plugin_redmine_git_hosting['gitUser']
-        @@old_gitolite_identity = Setting.plugin_redmine_git_hosting['gitoliteIdentityFile']
-	@@old_gitolite_publickey = Setting.plugin_redmine_git_hosting['gitoliteIdentityPublicKeyFile']
-	@@old_repo_base    = Setting.plugin_redmine_git_hosting['gitRepositoryBasePath']
-
+	@@old_valuehash = (Setting.plugin_redmine_git_hosting).clone
 
 	def reload_this_observer
 		observed_classes.each do |klass|
 			klass.name.constantize.add_observer(self)
 		end
 	end
 
+        # There is a long-running bug in ActiveRecord::Observer that prevents us from
+        # returning from before_save() with false to signal verification failure.
+        #
+        # Thus, we can only silently refuse to perform bad changes and/or perform
+        # slight corrections to badly formatted values.
         def before_save(object)
-        	if object.name == "plugin_redmine_git_hosting" && !GitHosting.bin_dir_writeable?
-                	# If bin directory not alterable, don't alow changes to
-          		# Git Username, or Gitolite public or private keys
+          	# Only validate settings for our plugin
+        	if object.name == "plugin_redmine_git_hosting" 
                 	valuehash = object.value
-	                valuehash['gitUser'] = @@old_git_user
-                  	valuehash['gitoliteIdentityFile'] = @@old_gitolite_identity
-                  	valuehash['gitoliteIdentityPublicKeyFile'] = @@old_gitolite_publickey
-			object.value = valuehash
+                	if !GitHosting.bin_dir_writeable?
+                        	# If bin directory not alterable, don't allow changes to
+          			# Git Username, or Gitolite public or private keys
+	                	valuehash['gitUser'] = @@old_valuehash['gitUser']
+                  		valuehash['gitoliteIdentityFile'] = @@old_valuehash['gitoliteIdentityFile']
+                  		valuehash['gitoliteIdentityPublicKeyFile'] = @@old_valuehash['gitoliteIdentityPublicKeyFile']
+                	end
+
+                  	# Normalize Repository path, should be relative and end in '/'
+                  	if valuehash['gitRepositoryBasePath']
+                        	normalizedFile  = File.expand_path(valuehash['gitRepositoryBasePath'].lstrip.rstrip,"/")
+                        	if (normalizedFile != "/")
+                        		valuehash['gitRepositoryBasePath'] = normalizedFile[1..-1] + "/"  # Clobber leading '/' add trailing '/'
+                        	else
+                        		valuehash['gitRepositoryBasePath'] = @@old_valuehash['gitRepositoryBasePath']
+                                end
+                        end
+
+                  	# Normalize Recycle bin path, should be relative and end in '/'
+                  	if valuehash['gitRecycleBasePath']
+                        	normalizedFile  = File.expand_path(valuehash['gitRecycleBasePath'].lstrip.rstrip,"/")
+                        	if (normalizedFile != "/")
+                        		valuehash['gitRecycleBasePath'] = normalizedFile[1..-1] + "/"  # Clobber leading '/' add trailing '/'
+                        	else
+                        		valuehash['gitRecycleBasePath'] = @@old_valuehash['gitRecycleBasePath']
+                        	end
+                        end
+                  
+                  	# Exclude bad expire times (and exclude non-numbers)
+			if valuehash['gitRecycleExpireTime']
+                        	if valuehash['gitRecycleExpireTime'].to_f > 0
+                                	valuehash['gitRecycleExpireTime'] = "#{(valuehash['gitRecycleExpireTime'].to_f * 10).to_i / 10.0}"
+	                        else
+                                	valuehash['gitRecycleExpireTime'] = @@old_valuehash['gitRecycleExpireTime']
+                                end
+                        end
+
+                  	# Validate wait time > 0 (and exclude non-numbers)
+                        if valuehash['gitLockWaitTime']
+                        	if valuehash['gitLockWaitTime'].to_i > 0
+                                	valuehash['gitLockWaitTime'] = "#{valuehash['gitLockWaitTime'].to_i}"
+                        	else
+                        		valuehash['gitLockWaitTime'] = @@old_valuehash['gitLockWaitTime']
+                                end
+                        end
+                	# Save back results
+                	object.value = valuehash
                 end
         end
 
 	def after_save(object)
+        	# Only perform after-actions on settings for our plugin
 		if object.name == "plugin_redmine_git_hosting"
+                	valuehash = object.value
 
                 	if GitHosting.bin_dir_writeable?
 				%x[ rm -rf '#{ GitHosting.get_tmp_dir }' ]
 				%x[ rm -rf '#{ GitHosting.get_bin_dir }' ] 
                         end
 
-			if @@old_repo_base != object.value['gitRepositoryBasePath']
+			if @@old_valuehash['gitRepositoryBasePath'] != valuehash['gitRepositoryBasePath']
 				GitHostingObserver.set_update_active(false)
 				all_projects = Project.find(:all)
 				all_projects.each do |p|
 					if p.repository.is_a?(Repository::Git)
 						r = p.repository
 						repo_name= p.parent ? File.join(GitHosting::get_full_parent_path(p, true),p.identifier) : p.identifier
-						r.url = File.join(object.value['gitRepositoryBasePath'], "#{repo_name}.git")
+						r.url = File.join(valuehash['gitRepositoryBasePath'], "#{repo_name}.git")
 						r.root_url = r.url
 						r.save
 					end
 				end
 				GitHostingObserver.set_update_active(true)
 			end
 
-			if @@old_git_user != object.value['gitUser']
+			if @@old_valuehash['gitUser'] != valuehash['gitUser']
 
 				GitHosting.setup_hooks
-				GitHosting.update_repositories( Project.find(:all), false)
+				GitHosting.update_repositories(:resync_all=>true)
 
-			elsif @@old_http_server !=  object.value['httpServer'] || @@old_hook_debug != object.value['gitHooksDebug'] || @@old_repo_base != object.value['gitRepositoryBasePath'] || @@old_hook_asynch != object.value['gitHooksAreAsynchronous']
+			elsif @@old_valuehash['httpServer'] !=  valuehash['httpServer'] || 
+                              @@old_valuehash['gitHooksDebug'] != valuehash['gitHooksDebug'] || 
+                              @@old_valuehash['gitRepositoryBasePath'] != valuehash['gitRepositoryBasePath'] || 
+                              @@old_valuehash['gitHooksAreAsynchronous'] != valuehash['gitHooksAreAsynchronous']
 
 				GitHosting.update_global_hook_params
 			end
-			@@old_hook_debug   = object.value['gitHooksDebug']
-			@@old_hook_asynch  = object.value['gitHooksAreAsynchronous']
-			@@old_http_server  = object.value['httpServer']
-			@@old_git_user     = object.value['gitUser']
-                        @@old_gitolite_identity = object.value['gitoliteIdentityFile']
-                        @@old_gitolite_publickey = object.value['gitoliteIdentityPublicKeyFile']
-			@@old_repo_base    = object.value['gitRepositoryBasePath']
 
+                  	@@old_valuehash = (Setting.plugin_redmine_git_hosting).clone
 		end
 	end
-
 end