Python进阶（下）

《Intermediate Python》阅读笔记下半部分

容器 Collections

一个模块

defaultdict：当字典里的key不存在但被查找时，返回的不是keyError而是一个默认值

from collections import defaultdict
dict = defaultdict( factory_function)  # 可以是list、set、str等，当key不存在时，返回工厂函数的默认值——[],(),'',0等

colours = ( ('Yasoob', 'Yellow'), 
           ('Ali', 'Blue'), 
           ('Arham', 'Green'), 
           ('Ali', 'Black'), 
           ('Yasoob', 'Red'), 
           ('Ahmed', 'Silver'), 
          )
favourite_colours = defaultdict(list)
for name, colour in colours:
    favourite_colours[name].append(colour)
# defaultdict(<type 'list'>, 
# {'Arham': ['Green'], 
# 'Yasoob': ['Yellow', 'Red'], 
# 'Ahmed': ['Silver'], 
# 'Ali': ['Blue', 'Black'] 
# })

counter：适用于需要使用dict和次数的场景，传入迭代器

from collections import Counter
colours = ( ('Yasoob', 'Yellow'), 
           ('Ali', 'Blue'), 
           ('Arham', 'Green'), 
           ('Ali', 'Black'), 
           ('Yasoob', 'Red'), 
           ('Ahmed', 'Silver'), 
          )
favs = Counter(name for name, colour in colours)
## Counter({ 
## 'Yasoob': 2, 
## 'Ali': 2,
## 'Arham': 1, 
## 'Ahmed': 1 
## })

c = Counter(a=3, b=1)
d = Counter(a=1, b=2)
c + d                       # 相加
#Counter({'a': 4, 'b': 3})

deque：双端队列，可以从头/尾两端添加或删除元素

from collections import deque
d = deque()
d = deque(range(5))
d.popleft()  # 输出: 0
d = deque(maxlen=30)  # 限制列表⼤⼩，当超出限制，数据会从队列另⼀端被挤出

nametuple：像字典(dict)⼀样访问namedtuples，但namedtuples不可变

from collections import namedtuple
Animal = namedtuple('Animal', 'name age type')
perry = Animal(name="perry", age=31, type="cat")
# Animal(name='perry', age=31, type='cat')
print(perry.name)  # 'perry'

枚举 Enumerate

内置函数

遍历数据并⾃动计数

for counter, value in enumerate(some_list):
	print(counter, value)
   
# 输出
(0, ...)
(1, ...)
...

定制从其他数字开始枚举

1	enumerate(my_list, 1) # 从1开始枚举

对象自省

运⾏时判断⼀个对象的类型的能⼒

dir

my_list = [1, 2, 3] 
dir(my_list)  # 返回⼀个列表对象的所有⽅法名
# Output: ['__add__', '__class__', '__contains__', '__delattr__', '__delitem__', 
# '__delslice__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', 
# '__getitem__', '__getslice__', '__gt__', '__hash__', '__iadd__', '__imul__', 
# '__init__', '__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__', 
# '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__rmul__', 
# '__setattr__', '__setitem__', '__setslice__', '__sizeof__', '__str__', 
# '__subclasshook__', 'append', 'count', 'extend', 'index', 'insert', 'pop', 
# 'remove', 'reverse', 'sort']

type与id：type函数返回⼀个对象的类型，id函数返回任意不同种类对象的唯⼀ID
1
2
3
4
5
print(type(''))
# Output: <type 'str'>
name = "Yasoob"
print(id(name))
# Output: 139972439030304

推导式

列表推导式

1	variable = [out_exp for out_exp in input_list if out_exp == 2]

字典推导式：类似列表推导式

mcase = {'a': 10, 'b': 34, 'A': 7, 'Z': 3}
# 同⼀个字母但不同⼤⼩写的值合并
mcase_frequency = {
    k.lower(): mcase.get(k.lower(), 0) + mcase.get(k.upper(), 0) for k in mcase.keys()
}

# 对换键值
{v: k for k, v in some_dict.items()}

集合推导式：使用大括号的”列表推导“
1
squared = {x**2 for x in [1, 1, 2]}

异常

try/except从句

处理多个异常

把所有可能发⽣的异常放到⼀个元组⾥

try:
    file = open('test.txt', 'rb')
except (IOError, EOFError) as e:
    print("An error occurred. {}".format(e.args[-1]))

对每个单独的异常在单独的except语句块中处理

捕获所有异常

try:
    file = open('test.txt', 'rb')
except Exception:
    # 打印⼀些异常⽇志，如果你想要的话
    raise

Finally从句：代码不管异常是否触发都将会被执⾏

try/else从句：在没有触发异常的时候执⾏⼀些代码

try:
    print('I am sure no exception is going to occur!') 
except Exception: 
    print('exception')
else:
    # 这⾥的代码只会在try语句⾥没有触发异常时运⾏, 
    # 但是这⾥的异常将 *不会* 被捕获 
    print('This would only run if no exception occurs. And an error here ' 'would NOT be caught.') 
finally: 
    print('This would be printed in every case.')

lambda表达式

是一行函数

lambda 参数:操作(参数)

add = lambda x, y: x + y
print(add(3, 5))

# 列表排序
a = [(1, 2), (4, 1), (9, 10), (13, -3)]
a.sort(key=lambda x: x[1])
# Output: [(13, -3), (4, 1), (1, 2), (9, 10)]
# 列表并行排序
data = zip(list1, list2)
data.sort()
list1, list2 = map(lambda t: list(t), zip(*data))

一行式

简易Web Server ：通过⽹络快速共享⽂件；在共享文件目录下的命令行输入
1
$ python -m http.server

规范的打印：规范打印出列表和字典

1
2
3

from pprint import pprint
my_dict = {'name': 'Yasoob', 'age': 'undefined'}
pprint(my_dict)

脚本性能分析：定位脚本中的性能瓶颈
1
$ python -m cProfile my_script.py

csv转为json

1	$ python -c "import csv,json;print json.dumps(list(csv.reader(open('csv_file.csv'))))"

列表展平

from itertools import itertools.chain.from_iterable
a_list = [[1, 2], [3, 4], [5, 6]]
print(list(itertools.chain.from_iterable(a_list))) 
# Output: [1, 2, 3, 4, 5, 6]

For else

for item in container: 
    if search_something(item): 
        # Found it! 
        process(item) 
        break 
else:  # else从句会在循环正常结束时执⾏，即循环没有遇到任何break
    # Didn't find anything.. 
    not_found_in_container()

使用C扩展

ctypes模块

提供了和C语⾔兼容的数据类型和函数来加载dll⽂件

示例

C代码：

#include <stdio.h> 
int add_int(int, int); 
float add_float(float, float); 
int add_int(int num1, int num2){
    return num1 + num2; 
}
float add_float(float num1, float num2){
    return num1 + num2; 
}

编译为.so文件（windows下为DLL）

#For windows
gcc -shared -o Test.dll Test.c
#For Linux 
$ gcc -shared -Wl,-soname,adder -o adder.so -fPIC add.c
#For Mac 
$ gcc -shared -Wl,-install_name,adder.so -o adder.so -fPIC add.c

python中调用

from ctypes import *
#load the shared object file 
adder = CDLL('./adder.so')
#Find sum of integers 
res_int = adder.add_int(4,5)

a = c_float(5.5) 
b = c_float(4.1)
add_float = adder.add_float
add_float.restype = c_float
add_float(a, b)

ctypes接口允许在调⽤C函数时使⽤Python中默认的字符串型和整型，对于其他布尔型和浮点型，应先转化为c_float
不能在C中对对象进⾏操作

Python/C API

以特定的⽅式来编写C代码
所有的Python对象都被表⽰为PyObject的结构体，且Python.h头⽂件中提供操作函数

示例

C代码

#include <Python.h>
static PyObject* addList_add(PyObject* self, PyObject* args){ 
    PyObject * listObj; 
   
    //The input arguments come as a tuple, we parse the args to get the various variables 
    //In this case it's only one list variable, which will now be referenced by listObj 
    if (! PyArg_ParseTuple( args, "O", &listObj )) 
        return NULL; 
    
    //length of the list long 
    length = PyList_Size(listObj); 
    
    //iterate over all the elements 
    int i, sum =0;
    for (i = 0; i < length; i++) { 
        //get an element out of the list - the element is also a python objects 
        PyObject* temp = PyList_GetItem(listObj, i); 
        //we know that object represents an integer - so convert it into C long 
        long elem = PyInt_AsLong(temp); 
        sum += elem; 
    }
    
    //value returned back to python code - another python object 
    //build value here converts the C long to a python integer 
    return Py_BuildValue("i", sum); 
}

//This is the docstring that corresponds to our 'add' function.
static char addList_docs[] = "add( ): add all elements of the list\n";

/* This table contains the relavent info mapping -
   <function-name in python module>, <actual-function>,
   <type-of-args the function expects>, <docstring associated with the function> 
*/ 
static PyMethodDef addList_funcs[] = { 
    {"add", (PyCFunction)addList_add, METH_VARARGS, addList_docs},
    {NULL, NULL, 0, NULL} 
};
/* addList is the module name, and this is the initialization block of the module. 
   <desired module name>, <the-info-table>, <module's-docstring> 
*/ 
PyMODINIT_FUNC initaddList(void){
    Py_InitModule3("addList", addList_funcs, "Add all ze lists"); 
} //模块初始化块签名为PyMODINIT_FUNC init{模块名}

setup.py：编译安装到Python模块

1
2
3

from distutils.core import setup, Extension
setup(name='addList', version='1.0', \
      ext_modules=[Extension('addList', ['adder.c'])])

1	$ python setup.py install

调用

1
2
3

import addList
l = [1,2,3,4,5]
print("Sum of List - " + str(l) + " = " + str(addList.add(l)))

上下文管理器

在有需要的时候，精确地分配和释放资源
最⼴泛的案例是with语句

基于类的实现

⼀个上下⽂管理器的类，最起码要定义__enter__和__exit__⽅法

class File(object): 
    def __init__(self, file_name, method): 
        self.file_obj = open(file_name, method) 
    def __enter__(self): 
        return self.file_obj 
    def __exit__(self, type, value, traceback): 
        print("Exception has been handled") 
        self.file_obj.close() 
        return True
        
with File('demo.txt', 'w') as opened_file: 
    opened_file.write('Hola!')

过程：
- with语句先暂存了File的__exit__
- 调⽤File的__enter__，打开⽂件并返回给with语句
- 使⽤.write()写⽂件
- with语句调⽤之前暂存的__exit__，关闭⽂件

异常处理

如果发⽣异常，异常的type,value和traceback传递给__exit__处理，如果__exit__返回的是True，这个异常被处理，否则with语句抛出异常