| |
Scrapy is an open-source opportunity to twisted the framework of python's stand-alone crawler, which actually contains most web crawler kits for the crawler download side as well as the extraction side.
CentOS 6.4 installation Scrapy 0.22 environmental process records, we hope to be helpful.
First, install Python2.7.6
1. Update the CentOS lib library file
Yum -y update
2. Install the Development Kit
Yum groupinstall -y development
3. Install the extension package
Yum install -y zlib-dev openssl-devel sqlite-devel bzip2-devel
4. Download Python 2.7.6
Wget http://www.python.org/ftp/python/2.7.6/Python-2.7.6.tar.xz
5. Unzip Python 2.7.6
Tar -xvf Python-2.7.6.tar.xz
6. Go to the Python 2.7.6 directory
Cd Python-2.7.6
7. Compile and install Python 2.7.6
./configure
Make && make altinstall
Second, solve Yum does not support Python2.7 problem
1. Back up the old Python program
Mv / usr / bin / python /usr/bin/python2.6.6
2. Create a new Python link
Ln -s /usr/local/bin/python2.7 / usr / bin / python
3. Modify the / usr / bin / yum file
Vi / usr / bin / yum
Change the first line to #! / Usr / bin / python to #! / Usr / bin / python2.6.6
Third, the installation setuptools and pip
1. Download setuptools
Wget --no-check-certificate https://pypi.python.org/packages/source/s/setuptools/setuptools-1.4.2.tar.gz
2. Unzip the setuptools
Tar -xvf setuptools-1.4.2.tar.gz
3. Enter the setuptools directory
Cd setuptools-1.4.2
4. Install setuptools from the installed Python2.7.6
Python setup.py install
5. Install pip
Easy_install pip
6.curl https://raw.github.com/pypa/pip/master/contrib/get-pip.py | python2.7 -
Fourth, the installation libxslt-devel support lxml
Yum install libxslt-devel
Fifth, install Scrapy
Pip install scrapy |
|
|
|