Home PC Games Linux Windows Database Network Programming Server Mobile  
           
  Home \ Programming \ Python2.7.7 source code analysis     - Docker data container protection the pros and cons (Server)

- Learning OpenCV: (VS2010-openCV2.4.3-win7 configuration instructions) (Linux)

- Java inner classes (Programming)

- The difference Docker save and export commands (Linux)

- Java Cookie Comments (Programming)

- lack of SWAP space during installation of Oracle (Database)

- High-performance JavaScript loops and flow control (Programming)

- Security matters and practical Linux System (Linux)

- Fedora 20 Installation and Configuration (Linux)

- TCP network communication Java Programming (Programming)

- RHEL (RedHat) 6.3 using EPEL Yum source (Linux)

- Briefly explain the invasion of the four-level denial of service attack DoS Linux server (Linux)

- Different versions of MongoDB achieve master-slave replication (Database)

- Ubuntu 14.10 / 14.04 how to install Quick Start tool Mutate 2.2 (Linux)

- C language print various graphic (Programming)

- Installation Flow N Play- interface stylized video player on Ubuntu (Linux)

- How to manage Vim plugin (Linux)

- Number JavaScript type system (Programming)

- Import and export myloader accelerate mydumper (Database)

- CentOS6.3 build a Python 3.3 environment to access Oracle 11gR2 (Database)

 
         
  Python2.7.7 source code analysis
     
  Add Date : 2018-11-21      
         
         
         
  As we all know, direct access to Python's official website is very slow, and may sometimes impossible to access. So easy to download the source code directly on here. Python 2.7.7 source code, learning Python reference material. tar.xz compression format. Use the following command to decompress:
tar -xvf ./Python-2.7.7.tar.xz

The main source directory structure

Demo: python sample program
Doc: Documentation
Grammar: The BNF syntax definition with all of Python's syntax, available to the parser
Include: header files used in the preparation of the expansion module with c / c ++
Lib: Python comes with standard libraries, written in python
Modules: using c prepared by the built-in module, zlib, md5 etc.
Objects: built-in object type implementations list, dict, etc.
PC: windows platform related documents
PCbuild: Microsoft Visual C ++ project directory
Parser: Python code for lexical analysis and parsing code
Python: byte-code compiler and interpreter
Tools: Some tools developed with Python
Object definition and creation of Python

In Python, everything is an object

Python object in fixed-length objects PyObject (such as int objects), variable length objects PyVarObject (such as a list object), the Python object belongs to one of these two. Object contains reference counts and types of information, management, and create the object needs to be used. Further comprising a storage property values.

PyObject object in memory is structured like the code below:

// In the Include / object.h in
// Length objects
typedef struct _object {
    int ob_refcnt; // reference counting for memory management
    struct _typeobject * ob_type; // type object containing type information
} PyObject;

// Variable length objects
typedef struct {
    int ob_refcnt; // reference counting for memory management
    struct _typeobject * ob_type; // type object containing type information
    // Variable length objects; Py_ssize_t ob_size (container classes: list, etc.) to accommodate the number of elements
} PyVarObject;

// Int objects
typedef struct {
    PyObject_HEAD
    long ob_ival;
} PyIntObject;
Type of object

In PyObject type of object (struct _typeobject * ob_type;), about the type of name, memory footprint size, structure and destructor function pointers and other attributes. These are the information needed to create the object.

// Define the type of object code snippets
typedef struct _typeobject {
    PyObject_VAR_HEAD
    / * Name, time printout format "< module> < name>." * /; Const char * tp_name
    Py_ssize_t tp_basicsize, tp_itemsize; / * create the object allocated memory size * /

    / * Standard way to achieve the type of * /
    destructor tp_dealloc;
    printfunc tp_print;
    getattrfunc tp_getattr;
    setattrfunc tp_setattr;
    cmpfunc tp_compare;
    reprfunc tp_repr;

    / * Operation numerical collection of objects * /
    PyNumberMethods * tp_as_number;

    / * Operation sequence collection of objects * /
    PySequenceMethods * tp_as_sequence;

    / * Method of operation associated with a collection of objects * /
    PyMappingMethods * tp_as_mapping;
// ...
} PyTypeObject;

// Python basic types of classes, all types should inherit from this type
PyAPI_DATA (PyTypeObject) PyType_Type; / * built-in 'type' * /
PyAPI_DATA (PyTypeObject) PyBaseObject_Type; / * built-in 'object' * /
PyAPI_DATA (PyTypeObject) PySuper_Type; / * built-in 'super' * /
 

All types of objects, custom class object types are PyType_Type (python type of built-in type), PyType_Type itself is a target.

The above PyBaseObject_Type class object is the base class for all other classes (python built-in object type). PyBaseObject_Type itself is an object.

Numerical methods of operation / sequence / collection of related objects, define the method of operation of these three objects have an object (such as a value object may also have [] the value operation).

Create object

Type object when the script is executed created and added to the symbol table. When you need to create an instance of the class object to get the type of object from the symbol table and create an instance of it (with associated symbol table code symtable.c file). C api to create a class object set of interfaces:

// Create an integer class object
PyObject * intObj = PyObject_New (PyObject, & PyInt_Type);

PyObject * _PyObject_New (PyTypeObject * tp)
{
    PyObject * op;
    op = (PyObject *) PyObject_MALLOC (_PyObject_SIZE (tp));
    if (op == NULL)
        return PyErr_NoMemory ();
    return PyObject_INIT (op, tp);
}
When you create an object in python, the creation process described in the following example:

class Test (object):
    pass

objTest = Test ()
PyObject_Call function is called, the argument is Test class object
Because inherited from the object, call the object type tp_call depending on the type of the object type
tp_call call tp_new, tp_init (corresponding to __init__ constructor python class initialization).

The built-in objects in python

python commonly used built-in objects are: integer objects, strings, lists, dictionaries object. These objects are used in the python up, so provide caching mechanism on the implementation to improve operational efficiency.

 

Integer object (PyIntObject)

After python integer objects are immutable (immutable), namely to create a python Integer object, you can not change the value of the object.

python to create an integer object provides the following three methods, which also calls PyInt_FromString and PyInt_FromUnicode internal Integer object PyInt_FromLong created.

PyObject * PyInt_FromLong (long ival);
PyObject * PyInt_FromString (char * s, char ** pend, int base);
PyObject * PyInt_FromUnicode (Py_UNICODE * s, Py_ssize_t length, int base);
The following major internal implementation facie PyInt_FromLong

PyObject * PyInt_FromLong (long ival)
{
    register PyIntObject * v;
#if NSMALLNEGINTS + NSMALLPOSINTS> 0
    // NSMALLNEGINTS = 5, NSMALLPOSINTS = 257
    // If the integer value of the object created in the [-5,256] returned directly from the cache pool small_ints Integer object
    if (-NSMALLNEGINTS < = ival && ival < NSMALLPOSINTS) {
        v = small_ints [ival + NSMALLNEGINTS];
        Py_INCREF (v);
#ifdef COUNT_ALLOCS
        if (ival> = 0)
            quick_int_allocs ++;
        else
            quick_neg_int_allocs ++;
#endif
        return (PyObject *) v;
    }
#endif
    // Create free_list cache list, provide to create the pool is not in the cache object small_ints
    if (free_list == NULL) {
        if ((free_list = fill_free_list ()) == NULL)
            return NULL;
    }
    // Get the new object from free_list
    v = free_list;
    free_list = (PyIntObject *) Py_TYPE (v);

    // Initializes and returns a new Integer object
    PyObject_INIT (v, & PyInt_Type);
    v-> ob_ival = ival;
    return (PyObject *) v;
}
Integer object destruction operation, only to release records in memory to free_list

static void
 int_dealloc (PyIntObject * v)
 {
     if (PyInt_CheckExact (v)) {
         Py_TYPE (v) = (struct _typeobject *) free_list;
         free_list = v;
     }
     else
         Py_TYPE (v) -> tp_free ((PyObject *) v);
 }
free_list PyIntBlock together with the management of small integers outside the range of the integer object cache

struct _intblock {
    struct _intblock * next;
    PyIntObject objects [N_INTOBJECTS];
};
typedef struct _intblock PyIntBlock;
static PyIntBlock * block_list = NULL;
static PyIntObject * free_list = NULL;
 

 

Implementation Mechanism Integer object summary

small_ints buffer pool is small integer object, the range [-5,256], you can quickly provide the object cache only increase object's reference count on it.

free_list and block_list save ever created Integer object allocated memory, when you create a new integer object directly from free_list object's memory space, can be used after the object is initialized.

python integer objects when released Integer object the memory will continue to be saved in block_list, and record free_list in the future, the newly created object to the use of an integer. (After you create an integer that is allocated memory is not returned to the operating system, so try to reduce the number of integer assigned the same time, which can reduce memory consumption)

 

 

String object

python string object is longer objects, but also objects immutable, the string can not be modified.

Inside are two ways to create a string object python, PyString_FromStringAndSize specified length.

PyObject * PyString_FromString (const char * str);
PyObject * PyString_FromStringAndSize (const char * str, Py_ssize_t size);
Look PyString_FromString internal implementation

PyObject * PyString_FromString (const char * str)
{
    register size_t size;
    register PyStringObject * op;

    assert (str = NULL!);
    size = strlen (str);
    if (size> PY_SSIZE_T_MAX - PyStringObject_SIZE) {
        PyErr_SetString (PyExc_OverflowError,
            "String is too long for a Python string");
        return NULL;
    }
    // Determine a single character can be returned directly from the cache
    if (size == 0 && (op = nullstring)! = NULL) {
#ifdef COUNT_ALLOCS
        null_strings ++;
#endif
        Py_INCREF (op);
        return (PyObject *) op;
    }
    if (size == 1 && (op = characters [* str & UCHAR_MAX])! = NULL) {
#ifdef COUNT_ALLOCS
        one_strings ++;
#endif
        Py_INCREF (op);
        return (PyObject *) op;
    }

    / * Inline PyObject_NewVar * /
    op = (PyStringObject *) PyObject_MALLOC (PyStringObject_SIZE + size);
    if (op == NULL)
        return PyErr_NoMemory ();
    PyObject_INIT_VAR (op, & PyString_Type, size);
    op-> ob_shash = -1;
    op-> ob_sstate = SSTATE_NOT_INTERNED;
    Py_MEMCPY (op-> ob_sval, str, size + 1);
    / * Create single-character buffer * /
    if (size == 0) {
        PyObject * t = (PyObject *) op;
        PyString_InternInPlace (& t);
        op = (PyStringObject *) t;
        nullstring = op;
        Py_INCREF (op);
    } Else if (size == 1) {
        PyObject * t = (PyObject *) op;
        PyString_InternInPlace (& t);
        op = (PyStringObject *) t;
        characters [* str & UCHAR_MAX] = op;
        Py_INCREF (op);
    }
    return (PyObject *) op;
}
Character buffer pool, the pool will be the first time create a single character when the object is filled, such as within the above PyString_FromString function.

#define UCHAR_MAX 0xff
static PyStringObject * characters [UCHAR_MAX + 1];
Performance-related '+' operator and join operations. Every time '+' operator will need to create a new object, and poor performance. join first calculate the total length of the result object, create a result string object, and then copy the data to a result memory location, so better performance.

static PyObject * string_concat (register PyStringObject * a, register PyObject * bb)
{
    // ...
    op = (PyStringObject *) PyObject_MALLOC (PyStringObject_SIZE + size);
    if (op == NULL)
        return PyErr_NoMemory ();
    PyObject_INIT_VAR (op, & PyString_Type, size);
    op-> ob_shash = -1;
    op-> ob_sstate = SSTATE_NOT_INTERNED;
    Py_MEMCPY (op-> ob_sval, a-> ob_sval, Py_SIZE (a));
    Py_MEMCPY (op-> ob_sval + Py_SIZE (a), b-> ob_sval, Py_SIZE (b));
    op-> ob_sval [size] = '\ 0';
    return (PyObject *) op;
}

static PyObject * string_join (PyStringObject * self, PyObject * orig)
{
    // ...
    // Calculate the total length of the string after splicing
    for (i = 0; i < seqlen; i ++) {
        const size_t old_sz = sz;
        item = PySequence_Fast_GET_ITEM (seq, i);
        if (! PyString_Check (item)) {
#ifdef Py_USING_UNICODE
            if (PyUnicode_Check (item)) {
                / * Defer to Unicode join * CAUTION:. There's no gurantee that the * original sequence can be iterated over * again, so we must pass seq here * /.
                PyObject * result;
                result = PyUnicode_Join ((PyObject *) self, seq);
                Py_DECREF (seq);
                return result;
            } #endif
            PyErr_Format (PyExc_TypeError,
                         "Sequence item% zd: expected string,"
                         "% .80s Found",
                         i, Py_TYPE (item) -> tp_name);
            Py_DECREF (seq);
            return NULL;
        }
        sz + = PyString_GET_SIZE (item);
        if (i! = 0)
            sz + = seplen;
        if (sz < old_sz || sz> PY_SSIZE_T_MAX) {
            PyErr_SetString (PyExc_OverflowError,
                "Join () result is too long for a Python string");
            Py_DECREF (seq);
            return NULL;
        }
    }

    // Allocate space for the string after splicing
    res = PyString_FromStringAndSize ((char *) NULL, sz);
    if (res == NULL) {
        Py_DECREF (seq);
        return NULL;
    }

    // Copy string concatenation to the newly created string memory location
    p = PyString_AS_STRING (res);
    for (i = 0; i < seqlen; ++ i) {
        size_t n;
        item = PySequence_Fast_GET_ITEM (seq, i);
        n = PyString_GET_SIZE (item);
        Py_MEMCPY (p, PyString_AS_STRING (item), n);
        p + = n;
        if (i < seqlen - 1) {
            Py_MEMCPY (p, sep, seplen);
            p + = seplen;
        }
    }

    Py_DECREF (seq);
    return res;}
 

Implementation Mechanism string object summary

Being Object String object to achieve a single-character buffer, to create an object when a single character from the buffer.

Use join splicing performance multi-string objects better than '+'.

Object list

Defined list of objects

list inside the object is to use an array to achieve, in an array of pointers, pointing to the object to be saved.

allocated in the list is the size of the array, ob_size is the current size of the array has been used.

typedef struct {
    // Variable length object has ob_size, save the array size currently being used
    PyObject_VAR_HEAD
    PyObject ** ob_item; // array of pointers
    Py_ssize_t allocated; // allocate array length
} PyListObject;
Cache object list

list object has a caching mechanism, the object upon release will be saved to the free buffer pool to be used when the next application. Buffer pool can cache 80 list object when the buffer pool full list objects directly released.

Know its caching mechanism created from the list of objects and destruction process (for focus, the code is simplified too).

// Define the size of the buffer pool
#define PyList_MAXFREELIST 80

// Create a new list objects
PyObject * PyList_New (Py_ssize_t size)
{
    PyListObject * op;
    size_t nbytes = size * sizeof (PyObject *);
    // If the cache has a free allocated memory list objects directly from the cache
    if (numfree) {
        numfree--;
        op = free_list [numfree];
        _Py_NewReference ((PyObject *) op);
    } Else {
        op = PyObject_GC_New (PyListObject, & PyList_Type);
        if (op == NULL)
            return NULL;
    }
    if (size < = 0)
        op-> ob_item = NULL;
    else {
        op-> ob_item = (PyObject **) PyMem_MALLOC (nbytes);
        if (op-> ob_item == NULL) {
            Py_DECREF (op);
            return PyErr_NoMemory ();
        }
        memset (op-> ob_item, 0, nbytes);
    }
    Py_SIZE (op) = size;
    op-> allocated = size;
    _PyObject_GC_TRACK (Op);
    return (PyObject *) op;
}

// Destroy the object list
static void list_dealloc (PyListObject * op)
{
    Py_ssize_t i;
    PyObject_GC_UnTrack (op);
    Py_TRASHCAN_SAFE_BEGIN (op)
    if (op-> ob_item! = NULL) {
        i = Py_SIZE (op);
        while (--i> = 0) {
            Py_XDECREF (op-> ob_item [i]);
        }
        PyMem_FREE (op-> ob_item);
    }
    // Save the list of objects to cache idle
    if (numfree < PyList_MAXFREELIST && PyList_CheckExact (op))
        free_list [numfree ++] = op;
    else
        Py_TYPE (op) -> tp_free ((PyObject *) op);
    Py_TRASHCAN_SAFE_END (op)
}
list of insert, delete, add operation

The realization of the internal list is an array, so the insert and delete operations will cause movement of internal elements. When you add the operation, if the current list object allocation memory is not used up, it is appended at the end directly.

Facie list insertion and add operations.

// Insert
int PyList_Insert (PyObject * op, Py_ssize_t where, PyObject * newitem)
{
    if (! PyList_Check (op)) {
        PyErr_BadInternalCall ();
        return -1;
    }
    return ins1 ((PyListObject *) op, where, newitem);
}

static int ins1 (PyListObject * self, Py_ssize_t where, PyObject * v)
{
    Py_ssize_t i, n = Py_SIZE (self);
    PyObject ** items;
    if (v == NULL) {
        PyErr_BadInternalCall ();
        return -1;
    }
    if (n == PY_SSIZE_T_MAX) {
        PyErr_SetString (PyExc_OverflowError,
            "Can not add more objects to list");
        return -1;
    }
    // Determine whether to re-allocation length
    if (list_resize (self, n + 1) == -1)
        return -1;

    // Find the insertion point
    if (where < 0) {
        where + = n;
        if (where < 0)
            where = 0;
    }
    if (where> n)
        where = n;

    // Move elements
    items = self-> ob_item;
    for (i = n; --i> = where;)
        items [i + 1] = items [i];
    Py_INCREF (v);
    items [where] = v;
    return 0;
}

// Add operation
int PyList_Append (PyObject * op, PyObject * newitem)
{
    if (PyList_Check (op) && (newitem! = NULL))
        return app1 ((PyListObject *) op, newitem);
    PyErr_BadInternalCall ();
    return -1;} static int app1 (PyListObject * self, PyObject * v) {
    Py_ssize_t n = PyList_GET_SIZE (self);

    assert (v = NULL!);
    if (n == PY_SSIZE_T_MAX) {
        PyErr_SetString (PyExc_OverflowError,
            "Can not add more objects to list");
        return -1;
    }

    if (list_resize (self, n + 1) == -1)
        return -1;

    Py_INCREF (v);
    PyList_SET_ITEM (self, n, v);
    return 0;
}
Object summary list

Internal quantitative target list caching to improve speed to create a list of objects

Insert the object list, delete the higher operating costs, not suitable for frequent operation.

append operation faster.

dict objects

Dict object definitions

Dict internal implementation object is the hash table, the hash function used in open address method, in theory, the time complexity of the algorithm is O (1).

dict object when the hash table is less than 8, the use of an array of objects ma_smalltable internal memory.

// Use the internal array space, create a smaller length of hash
#define PyDict_MINSIZE 8

// Hash table of data items
typedef struct {
    Py_ssize_t me_hash;
    PyObject * me_key;
    PyObject * me_value;} PyDictEntry; // dict objects struct _dictobject {
    PyObject_HEAD
    Count + pseudo deleted dummy count // use; Py_ssize_t ma_fill
    Py_ssize_t ma_used; // use count

    Py_ssize_t ma_mask;

    PyDictEntry * ma_table; // hash table memory pointer
    PyDictEntry * (* ma_lookup) (PyDictObject * mp, PyObject * key, long hash);
    // Internal memory optimization, the hash table is small; PyDictEntry ma_smalltable [PyDict_MINSIZE]
};
Dict object cache

dict objects have caching mechanism to the buffer pool when the object is released, to be used when the next application. Buffer pool can cache 80 dict object cache pool full time dict object directly released.

Know its caching mechanism created from dict objects and destruction process (for focus, the code is simplified too).

// Define the size of the buffer pool
#define PyDict_MAXFREELIST 80

// Create dict objects
PyObject * PyDict_New (void)
{
    register PyDictObject * mp;
    // Create a dummy object, use the time to remove placeholder
    if (dummy == NULL) {/ * Auto-initialize dummy * /
        dummy = PyString_FromString ( "< dummy key>");
        if (dummy == NULL)
            return NULL;
    }
    // Determine if there is idle cache, use cache dict objects
    if (numfree) {
        mp = free_list [- numfree];
        assert (mp = NULL!);
        assert (Py_TYPE (mp) == & PyDict_Type);
        _Py_NewReference ((PyObject *) mp);
        if (mp-> ma_fill) {
            EMPTY_TO_MINSIZE (mp);
        } Else {
            INIT_NONZERO_DICT_SLOTS (mp);
        }
        assert (mp-> ma_used == 0);
        assert (mp-> ma_table == mp-> ma_smalltable);
        assert (mp-> ma_mask == PyDict_MINSIZE - 1);
    } Else {
        mp = PyObject_GC_New (PyDictObject, & PyDict_Type);
        if (mp == NULL)
            return NULL;
        EMPTY_TO_MINSIZE (mp);
    }
    mp-> ma_lookup = lookdict_string;

    return (PyObject *) mp;
}

// Release dict function
static void dict_dealloc (register PyDictObject * mp)
{
    register PyDictEntry * ep;
    Py_ssize_t fill = mp-> ma_fill;
    PyObject_GC_UnTrack (mp);
    Py_TRASHCAN_SAFE_BEGIN (mp)
    for (ep = mp-> ma_table; fill> 0; ep ++) {
        if (ep-> me_key) {
            --fill;
            Py_DECREF (ep-> me_key);
            Py_XDECREF (ep-> me_value);
        }
    }
    if (mp-> ma_table! = mp-> ma_smalltable)
        PyMem_DEL (mp-> ma_table);
    // If there is free space in the cache, the cache released dict objects
    if (numfree < PyDict_MAXFREELIST && Py_TYPE (mp) == & PyDict_Type)
        free_list [numfree ++] = mp;
    else
        Py_TYPE (mp) -> tp_free ((PyObject *) mp);
    Py_TRASHCAN_SAFE_END (mp)
}
Open address hash table of the main search algorithm

dict objects hash search algorithm first compares the key is the same, not the same as the probe to the next position, until find elements or lookup fails. Find fail, return to the first available position.

static PyDictEntry * lookdict (PyDictObject * mp, PyObject * key, register long hash)
{
    register size_t i;
    register size_t perturb;
    register PyDictEntry * freeslot;
    register size_t mask = (size_t) mp-> ma_mask;
    PyDictEntry * ep0 = mp-> ma_table;
    register PyDictEntry * ep;
    register int cmp;
    PyObject * startkey;

    // Find the hash location
    i = (size_t) hash & mask;
    ep = & ep0 [i];
    if (ep-> me_key == NULL || ep-> me_key == key)
        return ep;

    // Determine whether the hash location placeholder objects after deletion
    if (ep-> me_key == dummy)
        freeslot = ep;
    else {
        // Hash hash matching, further search
        if (ep-> me_hash == hash) {
            startkey = ep-> me_key;
            Py_INCREF (startkey);
            cmp = PyObject_RichCompareBool (startkey, key, Py_EQ);
            Py_DECREF (startkey);
            if (cmp < 0)
                return NULL;
            if (ep0 == mp-> ma_table && ep-> me_key == startkey) {
                if (cmp> 0)
                    return ep;
            }
            else {
                return lookdict (mp, key, hash);
            }
        }
        freeslot = NULL;
    }

    // Find a match on the detection chain
    for (perturb = hash;; perturb >> = PERTURB_SHIFT) {
        i = (i < < 2) + i + perturb + 1;
        ep = & ep0 [i & mask];
        if (ep-> me_key == NULL)
            return freeslot == NULL ep:? freeslot;
        if (ep-> me_key == key)
            return ep;
        if (ep-> me_hash == hash && ep-> me_key! = dummy) {
            startkey = ep-> me_key;
            Py_INCREF (startkey);
            cmp = PyObject_RichCompareBool (startkey, key, Py_EQ);
            Py_DECREF (startkey);
            if (cmp < 0)
                return NULL;
            if (ep0 == mp-> ma_table && ep-> me_key == startkey) {
                if (cmp> 0)
                    return ep;
            }
            else {
                return lookdict (mp, key, hash);
            }
        }
        else if (ep-> me_key == dummy && freeslot == NULL)
            freeslot = ep;
    }
    return 0;
}
dict objects summary

dict objects with an open address hashing.

dict quantitative internal object cache to improve the speed to create a dict object.

For the length of the smaller dict objects, using the object internal memory directly, without secondary allocate memory, better performance.

What is bytecode

python interpreter when executing python script file in the file python source code is compiled, the result is compiled byte code (bytecode)

Python virtual machine running compiled bytecode to complete the program

python bytecode file is created for the module import

Creation bytecode files

When a.py rely b.py, as a.py in import b

python first check whether there b.pyc files (bytecode file), if there is, and modify later than b.py, directly call b.pyc

Otherwise, the compiler generates b.py b.pyc, then load the newly generated bytecode file

Bytecode objects

Each py file will contain many blocks of code (Code Block)

Each block of code (Code Block) will create a compiled bytecode objects (PyCodeObject)

PyCodeObject object itself is nested, according to the structure of the code block nested

Co_consts child PyCodeObject object stored in the variable in the parent object

Code blocks Code Block

The entire file # 1 is a block of code

Code block # 2
class TestA (object):
    pass

Code block # 3
class TestB (object):
    pass

Code block # 4
def show ():
    print 'show ...'

a = TestA () show ()
Structure bytecode objects

/ * Bytecode objects * /
typedef struct {
    PyObject_HEAD
    int co_argcount; / * Code Block number of parameters * /
    int co_nlocals; / * number of local variables * /
    int co_stacksize; / * stack space * /
    int co_flags; / * CO _..., see below * /
    PyObject * co_code; / * bytecode instruction sequence * /
    PyObject * co_consts; / * list of constants (subcode block is here) * /
    PyObject * co_names; / * String bulleted list * /
    PyObject * co_varnames; / * local variable name * /
    PyObject * co_freevars; / * closure variables needed * /
    PyObject * co_cellvars; / * local variable names nested functions referenced * /
    / * The rest does not count for hash / cmp * /
    PyObject * co_filename; / * py file path * /
    PyObject * co_name; / * Code block function name or class name * /
    int co_firstlineno; / * Code block starting line * /
    PyObject * co_lnotab; / * correspondence between bytecode instructions and source code * /
    void * co_zombieframe; / * for optimization only (see frameobject.c) * /
    PyObject * co_weakreflist; / * to support weakrefs to code objects * /
} PyCodeObject;
Realization of internal bytecode file

Bytecode object serialization result to the hard drive, is the bytecode files. This process is a process of recursive writing.

static void w_object (PyObject * v, WFILE * p) {
    Py_ssize_t i, n;
    p-> depth ++;

    / * ... * /
    else if (PyCode_Check (v)) {
        PyCodeObject * co = (PyCodeObject *) v;
        w_byte (TYPE_CODE, p);
        w_long (co-> co_argcount, p);
        w_long (co-> co_nlocals, p);
        w_long (co-> co_stacksize, p);
        w_long (co-> co_flags, p);
        w_object (co-> co_code, p);
        w_object (co-> co_consts, p);
        w_object (co-> co_names, p);
        w_object (co-> co_varnames, p);
        w_object (co-> co_freevars, p);
        w_object (co-> co_cellvars, p);
        w_object (co-> co_filename, p);
        w_object (co-> co_name, p);
        w_long (co-> co_firstlineno, p);
        w_object (co-> co_lnotab, p);
    }
    / * ... * /
}
Deserialization execute bytecode files

Bytecode bytecode object files are created in the deserialized structure, according to the virtual machine bytecode objects execution function at the time of execution.

In python, module, scope, name space these concepts and operational mechanism of virtual machines are closely connected,

Here to Learn module, scope, and name space for later analysis of running virtual machines to prepare.

module

In python is a file corresponds to a module, after being introduced into each py file corresponds to a module object.

This object contains a dict object that holds this py file corresponding variable and function references

Also holds a variable or function imported from another python file (module) reference.

Name space and scope

python has three separate name space, local, global, builtin.

builtin is a global name space python interpreter

global is a global name space module objects stored in the module's variables and functions references

local name space corresponding to the current block of code, save the local variables and function references


In the python file, each block of code corresponding to a scope, it corresponds to a name space.

When using a variable or function, the order of name space local, global, builtin search.

Code blocks can be nested, so the local name space is nested lookup.

Examples scope

#! / Usr / bin / env python
# Coding: utf-8
# Test.py
# Global scope, the definition of variable a, the function show

a = 'hello word'

def show ():
    # Local scope of function show
    name = 'show function'
    def myhelp ():
        # Function of the local scope myhelp
        # Use the global scope of a variable
        print a
        # Use the show function of the local scope variable name
        print name
to sum up

Code block corresponding to the scope and name space

Namespace preservation of variables and functions within the scope of reference

Each code block corresponding to a PyCodeObject objects pyc byte code file
     
         
         
         
  More:      
 
- Teach you to diagnose problems with strace (Linux)
- PPA on Ubuntu Linux installation Plank 0.8.0 (Linux)
- To disable the function by pressing Ctrl-Alt-Del to restart the system in RHEL / CentOS 5/6 (Linux)
- Bash added to the Vi mode indicator (Linux)
- Fedora 22 installation and configuration optimization (Linux)
- Oracle multi-user concurrency and transaction processing (Database)
- tar command to extract a file error: stdin has more than one entry (Linux)
- Weld a problem in JDK 8u60 environment (Programming)
- How to protect the Apache website under Linux system (Linux)
- SSH security note (Linux)
- Linux-du and df command (Linux)
- Build and verify MongoDB3.0.7 version (shard + replica) Cluster (Database)
- Performance issues under CentOS 6.5 VLAN devices (Linux)
- Python2.7.7 source code analysis (Programming)
- Windows SmartGit installation (Linux)
- Detailed installation of CentOS 6.x (Linux)
- Linux environment to configure Apache + Django + wsgi (Server)
- Use Mop monitor stock prices at the Linux command line (Linux)
- Zabbix API and PHP configuration (Server)
- To install and use Docker under linux (Server)
     
           
     
  CopyRight 2002-2022 newfreesoft.com, All Rights Reserved.