Home PC Games Linux Windows Database Network Programming Server Mobile  
           
  Home \ Linux \ The best tools and techniques to find data on a Unix system     - Boot automatically remove Linux operating history records (Linux)

- Learning how to teach safety system to prevent your own IP leakage (Linux)

- CentOS7 management of systemd (Linux)

- Linux Log (Linux)

- Postgres-X2 deployment steps (Database)

- To resolve Ubuntu 14.04 Unity Desktop Environment Login freeze problem (Linux)

- Repair after installing Ubuntu no boot device error (Linux)

- MySQL Statistics (Database)

- Linux system started to learn: how to solve the xxx is not in the sudoers file error (Linux)

- Quick Install software RAID on Linux (Linux)

- RMAN parameters of ARCHIVELOG DELETION (Database)

- The first deployment of cross-platform operation Rafy record (Server)

- Secondary exponential smoothing prediction method implemented in Python (Programming)

- Debian (Wheezy) Install Java environment / replace OpenJDK as the SUN JDK (Linux)

- Ubuntu dual-card system configuration method (Server)

- Ubuntu 14.04 Enable root and disable the guest (Linux)

- Linux shell script to adjust the Task Scheduler (Linux)

- Analysis of C ++ exception mechanism (Programming)

- Change all child files and subdirectories in the owner permissions Ubuntu (Linux)

- Use UDEV SCSI Rules configured ASM on Oracle Linux 5 and 6 (Database)

 
         
  The best tools and techniques to find data on a Unix system
     
  Add Date : 2017-01-08      
         
         
         
  Sometimes to find information on a Unix system is like needle in a haystack. If important information is lost in the large amount of text, they can hardly be noticed. Currently many of us who are dealing with "big data" - a collection of mining business intelligence from a few gigabytes of log files and the size of the huge variety of recording formats.

Fortunately, only in two cases, you only need to dig through piles of data, and then complete your work - when you know what you are looking for and when you do not know when. :) The best tools and techniques depending on your face two situations which one.

When you know when

When you know what you are looking for, grep is your friend, it's not just at the time you find specific text. grep command can help you find any of the text, a specific word, text mode and text has a context. When you know the text look like when it is usually easy to find. grep this that command will show "that" each line of the file contains the "this" string. Increase -w option will only show those lines alone contain "this" the word. In other words, if the line contains "thistle" or "erethism" it will not be revealed, unless these lines also have "this" word.

The easiest grep command does not charge any effort will be able to understand:

$ Grepfind poem
finding meaning, finding comfort,
finding someone to adore
Can we find a way to be
Find whole words can be completed by adding the -w option:

$ Grep-wfind poem
Can we find a way to be
Find mode requires a little skill. Our first example shows the line containing the "find" the word, regardless of "find" the "f" is uppercase or lowercase:

$ Grep [Ff] ind poem
Finding answers
finding meaning, finding comfort,
finding someone to adore
Can we find a way to be
If you want to match the text to the start or end of the line, you can use ^ (start) or $ (end).

$ Grep ^ find poem
finding meaning, finding comfort,
finding someone to adore
If you want to find the line that contains two consecutive vowels of words, you can use the "AEIOUaeiou" character as shown below.

$ Grep-E "[AEIOUaeiou] {2}" poem | head-3
Allour days are filled with searching
wondering what we're looking for
finding meaning, finding comfort,
Finds 9 or 10-letter string:

$ Grep-E "[[: alpha:]] {9,10}" poem
Allour days are filled with searching
wondering what we're looking for
All our days are filled with searching
that makes the searching more productive
Find contains a "find" of long words:

$ Grep-E "find [^ [: space:]] +" poem
finding meaning, finding comfort,
finding someone to adore
Most of us will not go to find poetry, this is obvious, but we can use the same techniques to obtain relevant information from our system files. In the following example, we find the "processor" of the term, and in accordance with a set of five elements (front two rows of rear two rows) are displayed in order to provide some context. If you want to get a 9-line group, -C 2 -C 4 into it.

$ Grep-C 2 processor / var / log / dmesg
Using ACPI (MADT) for SMP configuration information
Allocating PCI resources starting at 88000000 (gap: 80000000: 7ec00000)
Detected3400.426MHz processor.
Built1 zonelists.Total pages: 524275
Kernel command line: ro root = LABEL = / 1
-
Inode-cache hash table entries: 65536 (order: 6,262144 bytes)
Memory: 2071140k / 2097100k available (2223k kernel code, 24616k reserved, 922k data, 232kinit, 1179596k highmem)
Checkingifthis processor honours the WP bit even in supervisor mode ... Ok.
Calibrating delay loop (skipped), value calculated using timer frequency..6800.85BogoMIPS (lpj = 3400426)
SecurityFramework v1.0.0 initialized
-
CPU0: Intel (R) Xeon (TM) CPU 3.40GHz stepping 04
SMP alternatives: switching to SMP code
Booting processor 1/1 eip 11000
CPU 1 irqstacks, hard = c0779000 soft = c0759000
Initializing CPU
#1
-
CPU1: Intel (R) Xeon (TM) CPU 3.40GHz stepping 04
SMP alternatives: switching to SMP code
Booting processor 2/6 eip 11000
CPU 2 irqstacks, hard = c077a000 soft = c075a000
Initializing CPU
#2
-
CPU2: Intel (R) Xeon (TM) CPU 3.40GHz stepping 04
SMP alternatives: switching to SMP code
Booting processor 3/7 eip 11000
CPU 3 irqstacks, hard = c077b000 soft = c075b000
Initializing CPU
# 3

When you do not know when

If you want to find the text of a known location, such as when you tell Perl script execution to line 73 there is a problem, or you are dealing with the first 1892 lines of the file, you can use sed to display the specific line (I just do not like By 1892 the number of rows). And spend a little extra effort, you can display only the line.

The error message may look like this:

"Syntax error line 73 near"} else ""
You can use a sed command to display this line of problems:

$ Sed-n 73p showvars
else
Well, this is OK, but we do not know anything more than before. By displaying the first few lines can add a little context information, we can locate the error. There is a similar command can display ten lines of the line and before:

$ Sed-n 63,73p showvars
if $ password eq "a_secret";
{
foreach $ var (sort (keys (% ENV))) {
$ Val = $ ENV {$ var};
$ Val = ~ s | n | n | g;
$ Val = ~ s | "|" | g;
print '$ {var} = "$ {val}" n'
};
}
else
Whoops! It seems that some people writing if statement is out of the question! We can easily fix it.

You can also use the sed command lines to emphasize certain content. In the following commands, we added a "arrow" to highlight each row contains a foreach command:

$ Sed '/ print / {b label1; {: label1; s / ^ / # /; s / $ / <=== /;}}' showvars
#! / Bin / bash
if $ password eq "a_secret";
{
foreach $ var (sort (keys (% ENV))) {
$ Val = $ ENV {$ var};
$ Val = ~ s | n | n | g;
$ Val = ~ s | "|" | g;
#print '$ {var} = "$ {val}" n' <===
};
}
else
You can use a similar command commented your print command:

$ Sed '/ print / {b label1; {: label1; s / ^ / # /; s / $ / <=== /;}}' showvars
#! / Bin / bash
if $ password eq "a_secret";
{
foreach $ var (sort (keys (% ENV))) {
$ Val = $ ENV {$ var};
$ Val = ~ s | n | n | g;
$ Val = ~ s | "|" | g;
#print '$ {var} = "$ {val}" n' <===
};
}
else
Needle in a haystack is difficult, in fact, to find a needle on the carpet is also not easy. However, by using some of the most common Unix command deformation you can easily find what you're looking for something, even when you do not know what time looking.
     
         
         
         
  More:      
 
- A process of how to get the current traffic in GNU Linux (Linux)
- The wrong in Linux: too many open files (Linux)
- C ++ based foundation: the difference between C and C ++ (Programming)
- 30 Practical Linux system administrators will learn the command (Linux)
- DB2 commonly used scripting sort out (Database)
- Analyzing Linux server architecture is 32-bit / 64-bit (Server)
- The method of installing software under Ubuntu Linux (Linux)
- count (*) function in MySQL optimization of InnoDB storage engine (Database)
- Installation and configuration of Hadoop under Linux (Server)
- CentOS 6.5 installation and configuration Cobbler (Server)
- Linux system security check notes on performance (Linux)
- Using packet capture libpcap be reconciliation package in Ubuntu 14.04 64 bits (Linux)
- Change all child files and subdirectories in the owner permissions Ubuntu (Linux)
- What factors affect the performance of Java calls (Programming)
- Installation Elementary OS Freya 20 things to do (Linux)
- CentOS 6.5 installation VNCServer implement graphical access (Server)
- MySQL related statements (CRUD) (SQLyog software) (Database)
- Linux keyboard recording script (Linux)
- To deploy MySQL database with separate read and write OneProxy (Database)
- Python class of operator overloading (Programming)
     
           
     
  CopyRight 2002-2020 newfreesoft.com, All Rights Reserved.