How to split big files in to small chunks or in small parts ?

How to split big files in to small chunks or in small parts ?


What to do if you dont have much space on a drive to store any big file & you want to split and store it in to some small size parts.


A. First check md5sum hash of file, and make a note of it;

#md5sum <Big File Name>
e.g.
#md5sum Linux-pdf-HOWTOs.tar.bz2


B. Split file in to small managable chunks,

#split --bytes=<Size of each chunk>m <Big File Name> <Chunk File Prefix>
e.g.
#split --bytes=10m Linux-pdf-HOWTOs.tar.bz2 Linux-pdf-

  Note : Here it'll make 10-10 MB file chunks.


C. Now if you want to reassemble chunk files in to Big File,

#cat <File Chunk Prefix>* > <Big File Name>
e.g.
#cat Linux-pdf-a* > Linux_PDF_HOWTOS.tar.bz2


D. Check md5sum hash of file,

#md5sum <Big File Name>
e.g.
#md5sum Linux_PDF_HOWTOS.tar.bz2


E. Compare md5sum of "Step A" & "Step D", If both the md5sum are same then your big file is OK.

Comments

  1. Stancu Andrei SilviuJanuary 6, 2011 at 4:04 AM

    very nice .
    i once needed to backup a big files folder .. if i only knew about this guide .. lol ..
    anyways keep up the good work .

    ReplyDelete
  2. Data Recovery ServicesJune 13, 2012 at 1:02 PM

    Thanks for this fantastic info. I am looking for some help from your blog articles as I learn more about these things.
    data doctors houston

    ReplyDelete
  3. AnonymousMarch 19, 2015 at 6:23 PM

    Thank you

    ReplyDelete

Post a Comment

Popular posts from this blog

Recover or restore initramfs file in RHEL or CentOS 7

Image
What if "initramfs" file is deleted or corrupted on your RHEL or CentOS 7 ? Lets know what is initramfs image ? The INITial RAM Disk (initrd) is an initial root file system that usually get mounted prior to when the real root file system gets available. The "initrd" image is also known as "initramfs (Initial RAM FileSystem)" from RHEL 6.x onwards. The initrd/initramfs gets bound to the kernel and loaded as part of the kernel boot procedure. The kernel then mounts this initrd/initramfs as part of the two-stage boot process to load the modules to make the real file systems available and mount's the real root file system. Initrd/Initramfs image provides the capability to load a RAM disk by the boot loader. This RAM disk can then be mounted as the root filesystem and programs can be run from it. Afterwards, a new root file system can be mounted from a different device. The previous root filesystem which was mounted from initrd/initramfs is then
Read more

Space reclamation / UNMAP on RHEL or CentOS 7

What is Space Reclamation? This feature supports Thin Provisioned environment, and it is the process of freeing space from the storage system that has already been freed from the host file system. A host file system contains metadata to keep track to know which blocks are available to store new data and which blocks contain actual data and must not be overwritten. This metadata is stored within the LUN. When a file is deleted in the host file system, the file system metadata is updated to mark that file's blocks as free space. Total file system free space is then recalculated to include the newly-freed blocks. To the storage system, these metadata updates appear no different than any other writes being performed by the host. Therefore, the storage system is unaware that any deletions have occurred. What is Unmapping? Unmapping is a process which de-allocates relationship between LBA and a physical block in a logical unit. Reclamation within RHEL and CentOS: RHEL / Ce
Read more

How to recover /boot partition on RHEL or CentOS 7?

Image
What if mistakenly you delete all the files from /boot partition or files got corrupted due to some reason ! You will see below error on the screen at the time of system boot. But there is a way to recover /boot partition files :) Follow these steps to recover /boot partition : Step 1 :  Mount RHEL or CentOS 7 ISO image on your physical server and boot from it. In case you are using HPE Prolient server you can mount this ISO image on iLO, if this is virtual environment then mount it accordingly and reboot server or VM from ISO. Once rebooted you will see below options: Step 2 :  Scroll down and select "Troubleshooting" option from menu and PRESS "Enter" Step 3 :  Select "Rescue a CentOS system" and press ENTER: Step 4 :  Press ENTER key again to continue: Step 5 :  To continue, Type "1" and press ENTER, this will bring you to rescue mode, chroot sysimage filesystem using below command:
Read more