Thursday, March 23, 2017

Merge a Git Repo into Another Git Repo as a Branch

I have a "main" git repo that I use for most of our codebase (all of it, really). We recently begun an Android project, and had setup Android-Studio's VCS (version control system) incorrectly. About a dozen commits had been recorded in that system before we realized that we wanted to integrate it into our main repo.

I didn't want to use subtree or submodule, instead preferring to integrate it into a branch in the main repo. I also wanted to preserve whatever commit history was already in the VCS repo.

Here is how I did it...
(NOTE: make sure you're first checked-out in the new branch that you want to import/merge the VCS repo into, before continuing)

First, I went into the main repo (just using the git command line) and added the VCS repo as a remote:
# git remote add -f <remote-name> ssh://<user>@<remote-host>/path/to/androidStudio/asProject

Then, I did a fetch of that remote repo (the one with the Android project) into the main repo:
# git fetch <remote-name>

Next, I did a manual merge with the "ours" strategy (VCS just had one branch -master- in my case):
# git merge -s ours --no-commit <remote-name>/master

Next, you have to read all the stuff in:
(NOTE: this is where you can specify what subdirectory you want the "imported" repo to reside)
# git read-tree --prefix <main-repo/dest-path/asProj/> -u <remote-name>/master

Finally, you are ready to commit your the Android VCS repo into your main repo:
(you may do a status first, to double-check things before committing)
# git status
# git commit -m "adding existing Android Studio VCS repo to this new branch in main repo"

That's it!
To view your VCS commit history, just be sure the remote is visible in whatever GUI you use.

If you want to update the main repo with any changes in the Android Studio VCS repo (since you probably want to use that IDE in that environment), just pull it in with the "subtree" strategy:
# git pull -s subtree <remote-name> master

Tuesday, March 14, 2017

Creating an Image File for a Whole Hard Drive

In the course of my work, I often have a 2.5" SSD that needs to burned into an image for safe-keeping or later restoration. In my case, it's a 120GB drive that can be plugged into a simple SATA-USB adapter cable (powered by the USB bus). So, the hardest part is the software side... and it's really not that hard.

All you really need is the Linux 'dd' command and a compression utility. I've added a few minor things to make it faster and more informative.

First, ensure your distro has pigz (a gzip utility that can actually use multiple CPU cores for faster compression) and pv (pipe viewer, so you can monitor progress).

Then, plug in your drive, grab its device name from dmesg (I usually get /dev/sdi, so will use that in examples here), and run fdisk to get its size. That's optional, but nice to have for progress.

# dmesg
<get device id/name from output>

# fdisk -l /dev/sdi
<get size in bytes from output below, usually first line>
Disk /dev/sdi: 120.0 GB, 120034123776 bytes

Finally, issue the command to create an image file and plug in the size you got above to the pv portion of the command (of course, make sure you have enough space on disk for the compressed image, first!)...

# dd if=/dev/sdi | pv -s 120034123776 | pigz --fast > /path/to/new/imgFile.dd.gz

Wednesday, March 8, 2017

Mount a DD-created Disk Image

I have a lot of old disk images created with simple 'dd' commands. These images include the whole disk (all partitions) and are usually compressed. From time to time, I'll need to add files to those images or access data from them. Ideally, it's just quick to mount them to do that.

First, you need to decompress the image file. For gzip formats, I like to use pigz, since it's much faster on multi-core machines.

Now, you need to understand that you can only mount particular partitions. So, how do you access a particular partition in a single image file? You'll need to use a loopback offset.

Run fdisk to determine the partition table and get the data you'll need to calculate the offset.
# fdisk -l /path/to/IMG.dd
Disk IMG.dd: 120.0 GB, 120034123776 bytes
255 heads,63 sectors/track,14593 cylinders,tot 234441648 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0002beb4
 Device Boot      Start         End      Blocks   Id  System
IMG.dd1   *        2048   230336511   115167232   83  Linux
IMG.dd2       230338558   234440703     2051073    5  Extended
IMG.dd5       230338560   234440703     2051072   82  Linux swap

Now you need to calculate the offset. In my example above, I'm interested in partition #1, which fdisk reports to start at sector 2048. Since there are 512 bytes in each sector, as reported, multiply 512 X 2048 to get your offset value in bytes. In this case, it's 1048576. You'll use that number in the next step. Setup your loopback device offset, using the value you calculated above. I just used loop0, since it wasn't being used for anything on my system at the time. You should use whatever is available.
# losetup -o 1048576 /dev/loop0 /path/to/IMG.dd
Mount the loopback device.
# mount /dev/loop0 /mnt/img
Now you can use your filesystem as with any other file system you'd mount!

To unmount the file system, you'll need to unmount it like you normally would, but then you need to remove the loopback device.
# umount /mnt/img# losetup -d /dev/loop0