Home PC Games Linux Windows Database Network Programming Server Mobile  
           
  Home \ Linux \ The best tools and techniques to find data on a Unix system     - Linux install and configure Heartbeat (Server)

- Jigsaw project will solve the problem of Java JAR hell Mody (Programming)

- Java threads and thread pools (Programming)

- grub boot error resolution examples (Linux)

- Linux NIC driver and version information (Linux)

- MySQL completely uninstall and install Configuring Character Sets under Linux (Database)

- Linux scheduling summary (Linux)

- MySQL related statements (CRUD) (SQLyog software) (Database)

- Android Send HTTP POST requests (Programming)

- CentOS How to mount the hard drive (Linux)

- How to download apk file from the Google Play store on Linux (Linux)

- Installation and management of Linux applications (Linux)

- Oracle 11g DataGuard service to start automatically (Database)

- The Java Properties class action (Programming)

- hexdump Linux command (Linux)

- Turning off the interface eth0: error: Disconnect the device 'eth0' (Linux)

- CentOS 7 source code to compile and install Nginx process record (Server)

- Elaborate 10-point difference between the new and malloc (Programming)

- Linux processes in memory and memory cgroup statistics (Linux)

- Linux iptables firewall settings (Linux)

 
         
  The best tools and techniques to find data on a Unix system
     
  Add Date : 2017-01-08      
         
         
         
  Sometimes to find information on a Unix system is like needle in a haystack. If important information is lost in the large amount of text, they can hardly be noticed. Currently many of us who are dealing with "big data" - a collection of mining business intelligence from a few gigabytes of log files and the size of the huge variety of recording formats.

Fortunately, only in two cases, you only need to dig through piles of data, and then complete your work - when you know what you are looking for and when you do not know when. :) The best tools and techniques depending on your face two situations which one.

When you know when

When you know what you are looking for, grep is your friend, it's not just at the time you find specific text. grep command can help you find any of the text, a specific word, text mode and text has a context. When you know the text look like when it is usually easy to find. grep this that command will show "that" each line of the file contains the "this" string. Increase -w option will only show those lines alone contain "this" the word. In other words, if the line contains "thistle" or "erethism" it will not be revealed, unless these lines also have "this" word.

The easiest grep command does not charge any effort will be able to understand:

$ Grepfind poem
finding meaning, finding comfort,
finding someone to adore
Can we find a way to be
Find whole words can be completed by adding the -w option:

$ Grep-wfind poem
Can we find a way to be
Find mode requires a little skill. Our first example shows the line containing the "find" the word, regardless of "find" the "f" is uppercase or lowercase:

$ Grep [Ff] ind poem
Finding answers
finding meaning, finding comfort,
finding someone to adore
Can we find a way to be
If you want to match the text to the start or end of the line, you can use ^ (start) or $ (end).

$ Grep ^ find poem
finding meaning, finding comfort,
finding someone to adore
If you want to find the line that contains two consecutive vowels of words, you can use the "AEIOUaeiou" character as shown below.

$ Grep-E "[AEIOUaeiou] {2}" poem | head-3
Allour days are filled with searching
wondering what we're looking for
finding meaning, finding comfort,
Finds 9 or 10-letter string:

$ Grep-E "[[: alpha:]] {9,10}" poem
Allour days are filled with searching
wondering what we're looking for
All our days are filled with searching
that makes the searching more productive
Find contains a "find" of long words:

$ Grep-E "find [^ [: space:]] +" poem
finding meaning, finding comfort,
finding someone to adore
Most of us will not go to find poetry, this is obvious, but we can use the same techniques to obtain relevant information from our system files. In the following example, we find the "processor" of the term, and in accordance with a set of five elements (front two rows of rear two rows) are displayed in order to provide some context. If you want to get a 9-line group, -C 2 -C 4 into it.

$ Grep-C 2 processor / var / log / dmesg
Using ACPI (MADT) for SMP configuration information
Allocating PCI resources starting at 88000000 (gap: 80000000: 7ec00000)
Detected3400.426MHz processor.
Built1 zonelists.Total pages: 524275
Kernel command line: ro root = LABEL = / 1
-
Inode-cache hash table entries: 65536 (order: 6,262144 bytes)
Memory: 2071140k / 2097100k available (2223k kernel code, 24616k reserved, 922k data, 232kinit, 1179596k highmem)
Checkingifthis processor honours the WP bit even in supervisor mode ... Ok.
Calibrating delay loop (skipped), value calculated using timer frequency..6800.85BogoMIPS (lpj = 3400426)
SecurityFramework v1.0.0 initialized
-
CPU0: Intel (R) Xeon (TM) CPU 3.40GHz stepping 04
SMP alternatives: switching to SMP code
Booting processor 1/1 eip 11000
CPU 1 irqstacks, hard = c0779000 soft = c0759000
Initializing CPU
#1
-
CPU1: Intel (R) Xeon (TM) CPU 3.40GHz stepping 04
SMP alternatives: switching to SMP code
Booting processor 2/6 eip 11000
CPU 2 irqstacks, hard = c077a000 soft = c075a000
Initializing CPU
#2
-
CPU2: Intel (R) Xeon (TM) CPU 3.40GHz stepping 04
SMP alternatives: switching to SMP code
Booting processor 3/7 eip 11000
CPU 3 irqstacks, hard = c077b000 soft = c075b000
Initializing CPU
# 3

When you do not know when

If you want to find the text of a known location, such as when you tell Perl script execution to line 73 there is a problem, or you are dealing with the first 1892 lines of the file, you can use sed to display the specific line (I just do not like By 1892 the number of rows). And spend a little extra effort, you can display only the line.

The error message may look like this:

"Syntax error line 73 near"} else ""
You can use a sed command to display this line of problems:

$ Sed-n 73p showvars
else
Well, this is OK, but we do not know anything more than before. By displaying the first few lines can add a little context information, we can locate the error. There is a similar command can display ten lines of the line and before:

$ Sed-n 63,73p showvars
if $ password eq "a_secret";
{
foreach $ var (sort (keys (% ENV))) {
$ Val = $ ENV {$ var};
$ Val = ~ s | n | n | g;
$ Val = ~ s | "|" | g;
print '$ {var} = "$ {val}" n'
};
}
else
Whoops! It seems that some people writing if statement is out of the question! We can easily fix it.

You can also use the sed command lines to emphasize certain content. In the following commands, we added a "arrow" to highlight each row contains a foreach command:

$ Sed '/ print / {b label1; {: label1; s / ^ / # /; s / $ / <=== /;}}' showvars
#! / Bin / bash
if $ password eq "a_secret";
{
foreach $ var (sort (keys (% ENV))) {
$ Val = $ ENV {$ var};
$ Val = ~ s | n | n | g;
$ Val = ~ s | "|" | g;
#print '$ {var} = "$ {val}" n' <===
};
}
else
You can use a similar command commented your print command:

$ Sed '/ print / {b label1; {: label1; s / ^ / # /; s / $ / <=== /;}}' showvars
#! / Bin / bash
if $ password eq "a_secret";
{
foreach $ var (sort (keys (% ENV))) {
$ Val = $ ENV {$ var};
$ Val = ~ s | n | n | g;
$ Val = ~ s | "|" | g;
#print '$ {var} = "$ {val}" n' <===
};
}
else
Needle in a haystack is difficult, in fact, to find a needle on the carpet is also not easy. However, by using some of the most common Unix command deformation you can easily find what you're looking for something, even when you do not know what time looking.
     
         
         
         
  More:      
 
- Introduction to thread pooling and simple implementation (Programming)
- Docker Basic and Advanced (Linux)
- BackTrack (BT3, BT4) Linux installation tutorial (Linux)
- Try to use Lets Encrypt (Linux)
- Linux environment Duplex (multi-machine) automatic mutual backup scheme (Linux)
- Household use Linux Security (Linux)
- Linux uses the SMTP proxy to send mail (Linux)
- Mac OS X Server installation and application (Linux)
- Git and GitHub use of Eclipse and Android Studio (Programming)
- Understanding Java classes (Programming)
- Gentoo: startx problem appears Failed to load module (Linux)
- Memcached and Redis (Linux)
- Sublime Text 3 best features, plug-ins and settings (Linux)
- Ubuntu 14.04 LTS 64-bit installation and activation Sublime Text 3 can not solve the Chinese input method to solve the problem (Linux)
- Internal class broadcasting needs public and static (Programming)
- JQuery implements the same content merge cells (Programming)
- [Android] Eclipse does not update the Android SDK Manager solution [using GoAgent] (Programming)
- expdp reported ORA-39181 Export Processing Method (Database)
- Ubuntu 14.04.1 LTS compile and install the new kernel (Linux)
- Optimized to minimize the installation of CentOS 5.8 (Linux)
     
           
     
  CopyRight 2002-2022 newfreesoft.com, All Rights Reserved.