Migrate to Git LFS

Background

When I develop privately, I'll sometimes add large files into my repository. When I want to migrate the repo to Github, it creates problems. Enter Git Large File Storage (git lfs).

This page will show how I migrated a project with large files. Some of these files had been deleted in past past commits, but are still invisibly present in the repo.

Getting started

Since these actions will greatly modify your repo, making a backup is strongly recommended.

tar -jcvf myrepo.git.tar.bz2 myrepo.git

Ensure repo Integrity

mkdir /tmp/local && git init --bare /tmp/local
git remote add local /tmp/local
git push local

In my case, this failed:

$ git push local
fatal: bad tree object 8e7ebcd33a3237318f9324702d5793898ecc3ffe
error: remote unpack failed: eof before pack header was fully read
To /tmp/danger5
 ! [remote rejected] master -> master (unpacker error)
error: failed to push some refs to '/tmp/danger5'

If the remote is still available, you can fix this easily. You'll need a newer version of git (>=2.36). So let's update that if needed:

sudo apt update && sudo apt install software-properties-common
sudo add-apt-repository ppa:git-core/ppa
sudo apt install git

Then, in the problematic repo directory:

git fetch --refetch
git gc --aggressive

Once again try the push and see if your problem is resolved. If not, search it up!

Git Lfs

I recommend skimming through git-lfs tutorial

Install

sudo apt install git-lfs
# outside of any git repo:
git lfs install
# inside your git repo:
git lfs install

Migrate large files

In the case of migrating large files that are interspersed through the commits, you will need to use git lfs migrate. I figured out how to use it for the master branch with any file >50MB:

git lfs migrate import --above=50MB --include-ref=master

At this point, try pushing to a local-directory remote and testing things out.

Lastly, you can push to a fresh github remote repo.

git remote add github [email protected]:yourname/yourrepo.git
git push github

Cloning

You can try git lfs clone [repo]. If the lfs files have any problems, then you can try git lfs pull to get them manually.

References

How to Fix a Bad Git Tree Object