去年时，自己写一个程序时，不太确定自己的内存使用量，就想找写工具来打印程序或函数的内存使用量。
这里将上次找到的内存检测工具的基本用法记录一下，今后分析Python程序内存使用量时也是需要的。这里推荐2个工具吧。
memory_profiler模块（与psutil一起使用）
注：psutil这模块，我太喜欢了，它实现了很多Linux命令的主要功能，如：ps, top, lsof, netstat, ifconfig, who, df, kill, free 等等。
示例代码（https://github.com/smilejay/python/blob/master/py2014/mem_profile.py）：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
#!/usr/bin/env python ''' Created on May 31, 2014 @author: Jay <smile665@gmail.com> @description: use memory_profiler module for profiling programs/functions. ''' from memory_profiler import profile from memory_profiler import memory_usage import time @profile def my_func(): a = [1] * (10 ** 6) b = [2] * (2 * 10 ** 7) del b return a def cur_python_mem(): mem_usage = memory_usage(-1, interval=0.2, timeout=1) return mem_usage def f(a, n=100): time.sleep(1) b = [a] * n time.sleep(1) return b if __name__ == '__main__': a = my_func() print cur_python_mem() print "" print memory_usage((f, (1,), {'n': int(1e6)}), interval=0.5) |
运行上面的代码，输出结果为：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
jay@Jay-Air:~/workspace/python.git/py2014 $python mem_profile.py Filename: mem_profile.py Line # Mem usage Increment Line Contents ================================================ 15 8.0 MiB 0.0 MiB @profile 16 def my_func(): 17 15.6 MiB 7.6 MiB a = [1] * (10 ** 6) 18 168.2 MiB 152.6 MiB b = [2] * (2 * 10 ** 7) 19 15.6 MiB -152.6 MiB del b 20 15.6 MiB 0.0 MiB return a [15.61328125, 15.6171875, 15.6171875, 15.6171875, 15.6171875] [15.97265625, 16.00390625, 16.00390625, 17.0546875, 23.63671875, 23.63671875, 23.640625] |
Guppy (使用了Heapy）
Guppy is an umbrella package combining Heapy and GSL with support utilities such as the Glue module that keeps things together.
示例代码（https://github.com/smilejay/python/blob/master/py2014/try_guppy.py）：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
#!/usr/bin/env python ''' Created on May 31, 2014 @author: Jay <smile665@gmail.com> @description: just try to use Guppy-PE (useing Heapy) for memory profiling. ''' from guppy import hpy a = [8] * (10 ** 6) h = hpy() print h.heap() print h.heap().more print h.heap().more.more |
注意其中，要输出更多信息的.more用法。
运行上面的程序，输出结果为：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
jay@Jay-Air:~/workspace/python.git/py2014 $python try_guppy.py Partition of a set of 26963 objects. Total size = 11557848 bytes. Index Count % Size % Cumulative % Kind (class / dict of class) 0 177 1 8151560 71 8151560 71 list 1 12056 45 996840 9 9148400 79 str 2 5999 22 488232 4 9636632 83 tuple 3 324 1 283104 2 9919736 86 dict (no owner) 4 68 0 216416 2 10136152 88 dict of module 5 199 1 210856 2 10347008 90 dict of type 6 1646 6 210688 2 10557696 91 types.CodeType 7 1610 6 193200 2 10750896 93 function 8 199 1 177008 2 10927904 95 type 9 124 0 135328 1 11063232 96 dict of class <91 more rows. Type e.g. '_.more' to view.> Index Count % Size % Cumulative % Kind (class / dict of class) 10 1045 4 83600 1 11148456 96 __builtin__.wrapper_descriptor 11 109 0 69688 1 11218144 97 dict of guppy.etc.Glue.Interface 12 389 1 34232 0 11252376 97 __builtin__.weakref 13 427 2 30744 0 11283120 97 types.BuiltinFunctionType 14 411 2 29592 0 11312712 98 __builtin__.method_descriptor 15 25 0 26200 0 11338912 98 dict of guppy.etc.Glue.Share 16 108 0 25056 0 11363968 98 __builtin__.set 17 818 3 19632 0 11383600 98 int 18 66 0 18480 0 11402080 98 dict of guppy.etc.Glue.Owner 19 16 0 17536 0 11419616 99 dict of abc.ABCMeta <81 more rows. Type e.g. '_.more' to view.> （后面省略了部分输出） |
另外，还有一个叫“PySizer”的也是做memory profiling的，不过没怎么维护了。
我在这里只是根据文档举了几个最简单的例子，更多信息，请阅读如下参考文档：
stackoverflow.com/questions/110259/which-python-memory-profiler-is-recommended
https://pypi.python.org/pypi/memory_profiler
https://pypi.python.org/pypi/psutil
http://guppy-pe.sourceforge.net/