Jenkins is the tool we use for continuous integration and continuous delivery (TM) at OCF. All that means is that when you push code,
Ideally all projects at OCF will go through this pipeline of being tested before deployed, though currently some don't (or some only use some portion, such as deploying without any tests).
Anyone in group ocfroot can log in to Jenkins (using their OCF username and
password) and will have full access to Jenkins.
Sadly, while the installation of Jenkins is controlled via Puppet, its configuration is not. Configuring by Puppet would be nice, but it would mean changes would need to be made inside Puppet instead of the web UI.
In practice it seems most people in industry are still using the web UI for configuration anyway.
There are three users configured on the Jenkins server (reaper):
jenkins, the user created by the Debian package. It is used for running the
Jenkins master but not for performing any work.
jenkins-slave, a user we create. It is used for running build jobs with
potentially untrusted code. However, it's not secure enough to run
totally untrusted code, since all jobs run under this user.
jenkins-deploy, a user we create. It is used for running build jobs tagged
deploy, whose only purpose is intended to be deploying code which has
been built or tested in a previous step. The user has a Kerberos keytab for
the ocfdeploy user and our PyPI key in its home directory. Jobs such as
upload-deb or puppet-trigger fall under this user.
Within Jenkins, we configure two "slaves" which are really on the same server,
but execute by launching the slave.jar file as the jenkins-slave or
jenkins-deploy user (via passwordless sudo from the jenkins user,
effectively dropping permissions).
The jobs are configured to run on either jenkins-slave (the default) or
jenkins-deploy (for deploy jobs).
This is a bit complicated, but it allows us both better security (we no longer have to worry that anybody who can get some code built can become ocfdeploy, which is a privileged user account) and protects Jenkins somewhat against bad jobs that might e.g. delete files or crash processes.
Of course, in many cases once code builds successfully, we ship it off somewhere where it gets effectively run as root anyway. But this feels a little safer.
To test GitHub projects when you push to master:
Configure the "GitHub Project" URL to point to the main page of the project (for example, https://github.com/ocf/puppet/).
Under "Source Code Management", select "Git" and add the repository URL (for example, https://github.com/ocf/puppet/).
Under "Build Triggers", check "Build when a change is pushed to GitHub".
On GitHub, go to "Settings" then "Webhooks & services". Add a new "Jenkins
(GitHub Plugin)" service with URL
https://jenkins.ocf.berkeley.edu/github-webhook/.
You can create additional steps or organize pipelines if desired (for example, if you'd like to first test and then deploy).
You might like to add a fancy "Build Status" badge to the README. From the project page, choose the "Embeddable Build Status" icon, then choose "Markdown (with view), unprotected". You can optionally change the link to point to the pipeline view rather ther than just the individual job.
Jenkins can build and tag pull requests with their build status, similar to
Travis. To configure this for a repository, create a new job specifically for
testing pull requests. For example, puppet-test-pr.
Configure the "GitHub Project" URL to point to the main page of the project (for example, https://github.com/ocf/puppet/).
Under "Source Code Management", select "Git" and add the repository URL (for example, https://github.com/ocf/puppet/).
Under "Source Code Management", change "Branch Specifier" to ${sha1}.
Also under "Source Code Management", change "Refspec" (it's under Advanced)
to +refs/pull/*:refs/remotes/origin/pr/*.
Under "Build Triggers", check "GitHub Pull Request Builder", and then check "Use github hooks for build triggering".
Under "GitHub Pull Request Builder", delete all lines under "Admin List" (if there are any). Add "ocf" as the only line to the "List of organizations" box.
On GitHub, under "Settings" and "Webhooks & services", add a new webhook
with payload URL https://jenkins.ocf.berkeley.edu/ghprbhook/, content type
application/json, and the secret (it's in supernova:/opt/passwords).
Choose to trigger only on certain events:
(These might not all be necessary, but I don't know the exact list.)
On GitHub, add the "Bots" group admin access to the repository. This is necessary so that it can set commit statuses.