Atmospheric Spectroscopy Lab UMBC

Git Guidelines for ASL

Mon, Nov 9, 2015 L. Larrabee Strow

Guidelines for Using Git

I want everyone to start using git for work that they spend a signficant time on except (maybe) for interactive data plotting.

The basic point is that we need continuity in the group, ability to use one another’s software, and a record of how we obtained our results. This is especially true for climate studies where we process lot’s of information, but is also true for any/all analysis of data that we plan to report.

In addition, we need to start writing up pieces of our work in some standard way, that we can all view and change. That is the point of the private section of our ASL web site.

I am hoping that this gets us (and me) closer to writing up more of our work in journal articles.

What Goes in Git?

ASL Packages

In /asl/packages/git

  • sarta (one day)
  • kcarta matlab
  • kcarta fortran
  • rtp
  • klayers
  • oem package (not on Github yet!)
  • twp_proc (Andy’s Gruan analysis)
  • tsa, or time series analysis
  • sno generation
  • Howard has lot’s of examples already
  • PDF creation (maybe in rtp_prod2)
  • rtp_prod, matlib, etc.
  • aslhugo (web site)
  • ASL constants that we use over and over

Your Work Packages

In /asl/packages/git/name (already created)

  • oem_run (different names for different studies)
  • tsa_run
  • sno analysis
  • presentations
  • papers
  • LLS: DoFs for JPL for Ciras

.gitignore

Remember, git only stores what you tell it to store. Use .gitignore so git doesn’t keep bugging you about file types you don’t want under version control, like *.pdf, *.log, etc. Never do git add *. See Howard’s repos for good .gitignore files.

I do want you to store *.fig files, so I and others can easily look at your work.

Data Files

This is hard, but we need to have some standard places for larger data files that a git package needs. Github now has something called LFS (large file support), but if we use it, we can only (I think) use github for the repo. Local support for LFS is too early, too hard. However, I will probably use this for some projects (sarta, kcarta).

Small .mat files can go in the repo. Say, files 10 Mbytes or less, keeping repos under about 100 Mbytes or so. github will stop us at 1 Gbyte, I think.

I suggest that if you need data files for the package (of course, not including any of the big stuff on /asl/data) you put them in

/asl/s1/name/git_data/(directory tree of your choice)

What I really don’t want is packages that ever ever contain

/home/whoever/Work/Matlab/

or something like that. \home is NOT portable (Macs). Plus, I might back these directories up, and I need a place that is common to all to do that.

Github

In general, I like using github for most packages since it has a lot of nice, easy to use tool for examining changes, handling requests for merging to master, etc.

However, I don’t want to burden myself (I have to set them up) or you for every single time you want to create a repo. So, most of the time we will get started with saving the repo on maya. But, once there is traffic, we will move to github. You can always push to both repos.

Git Tools

At some point, you will want to learn how to grep in git to see diffs with older commits, etc. You will also want to learn what HEAD is, ^HEAD, etc.

Create Github Repo from Existing Repo

  1. Create a new repo on github. Do not initialize with READM, license, or gitinore files.
  2. Add the new github repo to the remotes for the existing repo. Something like git remote add repo_name git@github.com:strow/aslhugo.git.
  3. Push the existing repo to github, git push repo_name master

Note that when you do step 2. above, github will provide suggestions on how to push an existing repo to github, and they assume that the repo_name will be “origin”. Generally that is fine. But, for example, when I pushed aslhugo to github I used “github” for repo_name because I wanted to preserve “origin” for the repo that exists on maya.

I would suggest that you start putting your new repos on github on your personal accounts, and then as time goes on and they mature into packages, we will put them on github.com:strow.

Create a New Repo on Maya

Starting from Scratch

git init --bare --shared /asl/packages/git/rad_rates.git

Then locally (your directories)

git clone maya:/asl/packages/git/testlocal.git

Create file(s), git add them, git commit -m 'Initial Commit' and you should now have a master branch with something in it. You can then push using git push origin master.

Starting with an Existing Directory

This is the most likely scenario. Say you have a directory called twp_proc. First you do the same as above (with a new name):

git init --bare --shared /asl/packages/git/gruan_proc.git

Note the repo and the directory it is in can have different names. The, inside twp_proc:

git init
git add (files)
git commit -m 'Initial Commit'

and (this could be done earlier)

git remote add origin maya:/asl/packages/git/gruan_proc.git

git push origin master

and you are in business. Note I have maya defined in my .ssh/config file so I don’t have to type out the real name.

Make an Existing Repo Shared

I often forget the --shared qualifier when creating a repo. To fix that you need to cd into the repo on maya and do:

chgrp -R pi_strow .
find . -type d | xargs chmod g+ws
find . -type f | xargs chmod g+w

I think most of you should have permissions to do the above.

Git Flow for ASL

The ASL recommended approach for group development of packages is at our Git Flow page.