Home PC Games Linux Windows Database Network Programming Server Mobile  
           
  Home \ Linux \ QEMU image file used: qcow2 and raw     - Use custom backup plans for Debian backupninja (Linux)

- CentOS 6.6 compile and install phpMyAdmin configuration PostgreSQL9.4.1 (Database)

- CentOS build JDK environment (Linux)

- Grading defense against Linux server attacks (Linux)

- Vagrant failed to start, stuck in Waiting for VM to boot solution (Linux)

- Linux-based Heartbeat high availability configuration httpd service (Server)

- Slice MyCAT common fragmentation rules of enumeration (Database)

- Struts2 configure a static resource files without Struts processing (regular match) (Programming)

- First start with Kali Linux 2.0 (Linux)

- Get and Post requests Comments (Linux)

- The difference between Linux su and sudo commands (Linux)

- Install Redis 2.6 5.5 32 position CentOS error resolved (Linux)

- Install Websphere MB required system rpm package under Linux (Linux)

- Linux Workstation Security Checklist - from the Linux Foundation Internal (Linux)

- CentOS 6.5 installation VNCServer implement graphical access (Server)

- Oracle11g Trigger Debugging Record Error: PLS-00201: identifier SYS.DBMS_SYSTEM 'must be declared (Database)

- Build your own Web server under Ubuntu Linux system (Server)

- MySQL Study of --Percona Server 5.5 Upgrade 5.6 (Database)

- Ubuntu upgrade to Linux Kernel 4.2.3 Kernel (Linux)

- Reset CentOS / RHEL root account password 7 (Linux)

 
         
  QEMU image file used: qcow2 and raw
     
  Add Date : 2017-08-31      
         
         
         
  This article describes the qcow2 and raw, they are QEMU (KVM) virtual machine disk file format used in this article from its principle, support features, as well as reading and writing efficiency comparison and analysis and, finally, describes both formats how to transform a disk file.

The basic principle of qcow2

qcow2 image format is a QEMU emulator supports disk mirroring. It also can be used in the form of a file to indicate a fixed-size blocks of disk devices. Compared with ordinary raw image format has the following features:

A smaller footprint, even if the file system does not support hole (holes);
Support copy-on-write (COW, copy-on-write), image files only reflect changes in the underlying disk;
Support for snapshot (snapshot), image files can contain multiple snapshots of history;
Optional zlib based compression method
You can choose AES encryption
qcow2 image file format

Header

Each qcow2 file starts with a big-endian (big-endian) format head structured as follows:

Listing 1. qcow2 Header

 typedef struct QCowHeader {
      uint32_t magic;
      uint32_t version;

      uint64_t backing_file_offset;
      uint32_t backing_file_size;

      uint32_t cluster_bits;
      uint64_t size; / * in bytes * /
      uint32_t crypt_method;

      uint32_t l1_size;
      uint64_t l1_table_offset;

      uint64_t refcount_table_offset;
      uint32_t refcount_table_clusters;

      uint32_t nb_snapshots;
      uint64_t snapshots_offset;
  } QcowHeader;
Following a 10G of qcow2 file as an example to analyze the meaning of each field.

Listing 2. qcow2 hexadecimal file

# File 1.cow2
1.cow2: QEMU QCOW Image (v2), 10737418240 bytes

0000000: 5146 49fb 0000 0002 0000 0000 0000 0000 QFI .............
0000010: 0,000,000,000,000,010 0,000,000,280,000,000 ................
0000020: 0,000,000,000,000,014 0,000,000,000,030,000 ................
0000030: 0,000,000,000,010,000 0,000,000,100,000,000 ................
0000040: 0,000,000,000,000,000 0,000,000,000,000,000 ................
0000050: 0,000,000,000,000,000 0,000,000,000,000,000 ................
0000060: 0000 0004 0000 0068 0000 0000 0000 0000 ....... h ........
0000070: 0,000,000,000,000,000 0,000,000,000,000,000 ................
......
The first four bits contain the character Q, F, I, and then 0xfb, 5146 49fb instance is magic field.

The next four bits contain the version number of the image file, the instance is version 00000002 field representative using qcow2 version.

backing_file_offset 8 bytes, 0,000,000,000,000,000 example, given a start offset from a file.

backing_file_size gives a not-length null-terminated string, for instance 0000 0000. If the image file is a copy-writing, then it is the path of the original file.

cluster_bits, 32 bit (0000 0010), describes how to map the address of a mirror to a local file, it determines in a cluster, the lower offset address how as an index. Because the L2 table occupies a single cluster and contains an 8-byte entries (entry), so cluster_bits fewer than three bits as an index L2 table.

The next size, 8 bytes represent the size of the block device image file represented by the example of 0,000,000,280,000,000 bytes, which is the 10G space.

crypt_method If AES encryption for 1 representative.

l1_size (0000 0014) and l1_table_offset (0000 0000 0003 0000: :) are given in Table L1 size and offset.

refcount_table_offset given refcount table offset (0,000,000,000,010,000) and refcount_table_clusters describes the size (0000 0001) to cluster units refcount table.

nb_snapshots gives the number of snapshots (0000 0000) the image contains, snapshots_offset Each snapshot is given to offset the QCowSnapshotHeader (0,000,000,000,000,000).

A typical qcow2 image file contains about several parts:

Header information mentioned above
L1 Table
refcount table
One or more blocks refcount
Snapshot head
L2 table
Data cluster
Level 2 Find

In qcow2, the contents of the disk is stored in the cluster (each cluster contains some of the sector size of 512 bytes). To find the cluster where the given address, we need to find two tables, L1-> L2. L1 L2 table holds a set of tables to offset, L2 table to save a set of cluster offset;

Therefore, an address to be divided into three parts according to cluster_bits (64-bit) is set, for example cluster_bits = 12;

Low 12 is a 4Kb cluster offset (2 to the 12th power = 4Kb);

The next nine are contained in Table 512 L2 sheet items;

The remaining 43 delegates L1 table offset.

In order to obtain a given address (64) offset position:

L1 table address obtained from l1_table_offset Head domain
Before use (64-l2_bits-cluster_bits) L1-bit address to the index table
In the L1 table to obtain offset address L2 table
With the address of the next l2_bits index L2 table to get a 64-bit entries
Cluster obtained with L2 table offset address
With the remaining cluster_bits address bits to the index of the cluster, obtain the data block
If L1 and L2 table offset tables are empty, this area has not yet been assigned to the image file.

Note Table L1 and L2 table offset of the first two are retained and used as showing 'copied' or 'compressed'.

Copy-on-Write image file

qcow2 image can be used to save another image file changes, it is not to modify the original image file, only with a different record to the original image file, this image file is called copy-on-write image. Although it is a separate file, but it's most of the data from the original image, the original image file, compared with only a change in the cluster will be recorded.

It's very easy to achieve, the path to the original file recorded in the header information. When you need to read from a copy-on-write image file in a cluster, the first thing to check whether this area has already been allocated in the image file, if not read from the original file.

Snapshot

Snapshot somewhat similar Copy-On-Write files, but the difference is that a snapshot is writable. Snapshot is the original file itself (internal snapshot). It contains the original file before partially snapshots, which itself contains a writable portion.

Each snapshot contains the following header structure:

Listing 3. qcow2 snapshot Header

  typedef struct QCowSnapshotHeader {
      / * Header is 8 byte aligned * /
      uint64_t l1_table_offset;

      uint32_t l1_size;
      uint16_t id_str_size;
      uint16_t name_size;

      uint32_t date_sec;
      uint32_t date_nsec;

      uint64_t vm_clock_nsec;

      uint32_t vm_state_size;
      uint32_t extra_data_size; / * for extension * /
      / * Extra data follows * /
      / * Id_str follows * /
      / * Name follows * /
  } QcowSnapshotHeader;
Other features of qcow2

qcow2 support compression, which allows each cluster (cluster) alone zlib compression. It also supports 128-bit AES key encryption.

Create contrast qcow2 and raw files as well as two mirrors

Use QEMU package comes with software to create qcow2 qemu-img file.

Listing 4. Create qcow2 and raw files

$ Qemu-img create -f qcow2 test.qcow2 10G
Formatting 'test.qcow2', fmt = qcow2 size = 10737418240 encryption = off cluster_size = 65536 lazy_refcounts = off

$ Qemu-img create -f raw test.raw 10G
Formatting 'test.raw', fmt = raw size = 10737418240
Compare the actual size and space the size of two file formats are as follows:

Listing 5. qcow2 and raw files take up space contrast situations

$ Ll -sh test. *
200K -rw-r - r-- 1 peterhe peterhe 193K 5 Yue 6 10:29 test.qcow2
   0 -rw-r - r-- 1 peterhe peterhe 10G 5 Yue 6 10:28 test.raw

[peterhe @ localhost] $ stat test.raw
  File: "test.raw"
  Size: 10,737,418,240: 0 IO Block: 4096 regular file

[peterhe @ localhost] $ stat test.qcow2
  File: "test.qcow2"
  Size: 197 120: 400 IO Block: 4096 regular file
As can be seen from the comparison qcow format image file size is 197,120 bits byte space is 200K, taking up disk space 200. The file format is not raw disk space, it is an empty file.

Raw format conversion and qcow2

qemu-img tool QEMU package which provides a mirror image can be used for common operations.

The raw format into a file format qcow2 command as follows:

qemu-img convert -f raw -O qcow2 test.raw test.raw.qcow2
[peterhe @ localhost kimchi] $ ll -sh test. *
200K -rw-r - r-- 1 peterhe peterhe 193K 5 Yue 6 10:29 test.qcow2
   0 -rw-r - r-- 1 peterhe peterhe 10G 5 Yue 6 10:28 test.raw
200K -rw-r - r-- 1 peterhe peterhe 193K 5 Yue 6 10:44 test.raw.qcow2
Compare performance of the two file formats

Table 1. Comparison ide as a virtual disk drive performance three image formats

cache = off writethrough writeback
Old qcow2 (0.10.5) 16:52 min 28:58 min 6:02 min
New qcow2 (0.11.0-rc1) 5:44 min 9:18 min 6:11 min
raw 5:41 min 7:24 min 6:03 min
Table 2. Performance comparison using three kinds of image formats virtio as a virtual disk drive

cache = off writeback
Old qcow2 (0.10.5) 31:09 min 8:00 min
New qcow2 (0.11.0-rc1) 18:35 min 8:41 min
raw 8:48 min 7:51 min
summary

This article focuses on the QEMU virtual machine image file format used qcow2 and characteristics, and with the raw image format to do a comparison. qcow2 file formats, although some loss in performance than rRaw format (mainly reflected in the incremental file, the file format in order to allocate cluster qcow2 spent some more time), but the mirror qcow2 format smaller than the Raw file format only when the virtual machine actually taking up disk space, the file will grow, can easily reduce the flow of migration to spend more suitable for cloud computing systems, at the same time, it also has the encryption, compression, snapshots, and does not have a raw format function.
     
         
         
         
  More:      
 
- How to use scientific data Docker quickly configure the development environment (Server)
- MySQL5.7.10 installation documentation (Database)
- Linux crontab (Linux)
- Linux System Getting Started Tutorial: How do you know Shell which is currently being used (Linux)
- Installation and Configuration ISC DHCP server on Debian Linux (Server)
- Android webView URL redirects affect goBack () (Programming)
- Oracle 11G using DG Broker create DataGuard (Database)
- Storm how to assign tasks and load balancing (Programming)
- CentOS 7 server environment to quickly build Linux (Server)
- Ubuntu Control Panel to resolve network-manager icon display issue (Linux)
- Linux centos rm directory does not prompt (Linux)
- Ubuntu system grub repair method (Linux)
- To read the Linux ext3 / ext4 format partitions under Windows system software (Linux)
- The difference Docker save and export commands (Linux)
- MySQL event table to achieve timing build a small note (Database)
- Linux data redirection (Linux)
- Linux script commands - terminal recorder (Linux)
- CentOS 6.5 / Linux restart network card being given Determining if ip address x.x.x.x is already in use (Linux)
- RedHat 6.5 installation and deployment Openfire (Server)
- Linux Network Programming - raw socket instance: MAC Address Scanner (Programming)
     
           
     
  CopyRight 2002-2022 newfreesoft.com, All Rights Reserved.