自己动手写一个简易对象关系映射,的属性查找

图片 19

准备知识

原文出处: xybaby   

DBUtils模块  <<—–重点

在Python中,属性查找(attribute
lookup)是比较复杂的,特别是涉及到描述符descriptor的时候。

DBUtils是Python的一个用于实现数据库连接池的模块

此连接池有两种连接模式:

    DBUtils提供两种外部接口:
    PersistentDB :提供线程专用的数据库连接,并自动管理连接。
    PooledDB :提供线程间可共享的数据库连接,并自动管理连接。

在上一文章末尾,给出了一段代码,就涉及到descriptor与attribute
lookup的问题。而get系列函数(__get__, __getattr__,
__getattribute__) 也很容易搞晕,本文就这些问题简单总结一下。

图片 1图片 2

首先,我们知道:

from DBUtils.PooledDB import PooledDB
import pymysql

POOL = PooledDB(
    creator=pymysql,  # 使用链接数据库的模块
    maxconnections=6,  # 连接池允许的最大连接数,0和None表示不限制连接数
    mincached=2,  # 初始化时,链接池中至少创建的空闲的链接,0表示不创建
    maxcached=5,  # 链接池中最多闲置的链接,0和None不限制
    maxshared=3,  # 链接池中最多共享的链接数量,0和None表示全部共享。PS: 无用,因为pymysql和MySQLdb等模块的 threadsafety都为1,所有值无论设置为多少,_maxcached永远为0,所以永远是所有链接都共享。
    blocking=True,  # 连接池中如果没有可用连接后,是否阻塞等待。True,等待;False,不等待然后报错
    maxusage=None,  # 一个链接最多被重复使用的次数,None表示无限制
    setsession=[],  # 开始会话前执行的命令列表。
    ping=0,
    # ping MySQL服务端,检查是否服务可用。
    host='127.0.0.1',
    port=3306,
    user='root',
    password='123456',
    database='youku',
    charset='utf8',
    autocommit = True
)
  • python中一切都是对象,“everything is
    object”,包括类,类的实例,数字,模块
  • 任何object都是类(class or type)的实例(instance)
  • 如果一个descriptor只实现了__get__方法,我们称之为non-data
    descriptor, 如果同时实现了__get__ __set__我们称之为data
    descriptor。

DBUtils,配置模板

实例属性查找

按照python
doc,如果obj是某个类的实例,那么obj.name(以及等价的getattr(obj,’name’))首先调用__getattribute__。如果类定义了__getattr__方法,那么在__getattribute__抛出
AttributeError
的时候就会调用到__getattr__,而对于描述符(__get__)的调用,则是发生在__getattribute__内部的。官网文档是这么描述的

The implementation works through a precedence chain that gives data
descriptors priority over instance variables, instance variables
priority over non-data descriptors, and assigns lowest priority to
__getattr__()
if provided.

obj = Clz(), 那么obj.attr 顺序如下:

(1)如果“attr”是出现在Clz或其基类的__dict__中, 且attr是data
descriptor, 那么调用其__get__方法, 否则

(2)如果“attr”出现在obj的__dict__中, 那么直接返回
obj.__dict__[‘attr’], 否则

(3)如果“attr”出现在Clz或其基类的__dict__中

(3.1)如果attr是non-data descriptor,那么调用其__get__方法, 否则

(3.2)返回 __dict__[‘attr’]

(4)如果Clz有__getattr__方法,调用__getattr__方法,否则

(5)抛出AttributeError

下面是测试代码:

#coding=utf-8 class DataDescriptor(object): def __init__(self,
init_value): self.value = init_value def __get__(self, instance,
typ): return ‘DataDescriptor __get__’ def __set__(self,
instance, value): print (‘DataDescriptor __set__’) self.value =
value class NonDataDescriptor(object): def __init__(self,
init_value): self.value = init_value def __get__(self, instance,
typ): return(‘NonDataDescriptor __get__’) class Base(object):
dd_base = DataDescriptor(0) ndd_base = NonDataDescriptor(0) class
Derive(Base): dd_derive = DataDescriptor(0) ndd_derive =
NonDataDescriptor(0) same_name_attr = ‘attr in class’ def
__init__(self): self.not_des_attr = ‘I am not descriptor attr’
self.same_name_attr = ‘attr in object’ def __getattr__(self, key):
return ‘__getattr__ with key %s’ % key def change_attr(self):
self.__dict__[‘dd_base’] = ‘dd_base now in object dict ‘
self.__dict__[‘ndd_derive’] = ‘ndd_derive now in object dict ‘
def main(): b = Base() d = Derive() print ‘Derive object dict’,
d.__dict__ assert d.dd_base == “DataDescriptor __get__” assert
d.ndd_derive == ‘NonDataDescriptor __get__’ assert d.not_des_attr
== ‘I am not descriptor attr’ assert d.no_exists_key ==
‘__getattr__ with key no_exists_key’ assert d.same_name_attr ==
‘attr in object’ d.change_attr() print ‘Derive object dict’,
d.__dict__ assert d.dd_base != ‘dd_base now in object dict ‘
assert d.ndd_derive == ‘ndd_derive now in object dict ‘ try:
b.no_exists_key except Exception, e: assert isinstance(e,
AttributeError) if __name__ == ‘__main__’: main()

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
#coding=utf-8
class DataDescriptor(object):
    def __init__(self, init_value):
        self.value = init_value
 
    def __get__(self, instance, typ):
        return ‘DataDescriptor __get__’
 
    def __set__(self, instance, value):
        print (‘DataDescriptor __set__’)
        self.value = value
 
class NonDataDescriptor(object):
    def __init__(self, init_value):
        self.value = init_value
 
    def __get__(self, instance, typ):
        return(‘NonDataDescriptor __get__’)
 
class Base(object):
    dd_base = DataDescriptor(0)
    ndd_base = NonDataDescriptor(0)
 
 
class Derive(Base):
    dd_derive = DataDescriptor(0)
    ndd_derive = NonDataDescriptor(0)
    same_name_attr = ‘attr in class’
 
    def __init__(self):
        self.not_des_attr = ‘I am not descriptor attr’
        self.same_name_attr = ‘attr in object’
 
    def __getattr__(self, key):
        return ‘__getattr__ with key %s’ % key
 
    def change_attr(self):
        self.__dict__[‘dd_base’] = ‘dd_base now in object dict ‘
        self.__dict__[‘ndd_derive’] = ‘ndd_derive now in object dict ‘
 
def main():
    b = Base()
    d = Derive()
    print ‘Derive object dict’, d.__dict__
    assert d.dd_base == "DataDescriptor __get__"
    assert d.ndd_derive == ‘NonDataDescriptor __get__’
    assert d.not_des_attr == ‘I am not descriptor attr’
    assert d.no_exists_key == ‘__getattr__ with key no_exists_key’
    assert d.same_name_attr == ‘attr in object’
    d.change_attr()
    print ‘Derive object dict’, d.__dict__
    assert d.dd_base != ‘dd_base now in object dict ‘
    assert d.ndd_derive == ‘ndd_derive now in object dict ‘
 
    try:
        b.no_exists_key
    except Exception, e:
        assert isinstance(e, AttributeError)
 
if __name__ == ‘__main__’:
    main()

注意第50行,change_attr给实例的__dict__里面增加了两个属性。通过上下两条print的输出如下:

  Derive object dict {‘same_name_attr’: ‘attr in object’,
‘not_des_attr’: ‘I am not descriptor attr’}

Derive object dict {‘same_name_attr’: ‘attr in object’,
‘ndd_derive’: ‘ndd_derive now in object dict ‘, ‘not_des_attr’: ‘I
am not descriptor attr’, ‘dd_base’: ‘dd_base now in object dict ‘}

调用change_attr方法之后,dd_base既出现在类的__dict__(作为data
descriptor), 也出现在实例的__dict__, 因为attribute
lookup的循序,所以优先返回的还是Clz.__dict__[‘dd_base’]。而ndd_base虽然出现在类的__dict__,
但是因为是nondata
descriptor,所以优先返回obj.__dict__[‘dd_base’]。其他:line48,line56表明了__getattr__的作用。line49表明obj.__dict__优先于Clz.__dict__

def func():
    ...
    conn = POOL.connection()
    ...

cached_property例子

我们再来看看上一文章的这段代码。

import functools, time class cached_property(object): “”” A property
that is only computed once per instance and then replaces itself with an
ordinary attribute. Deleting the attribute resets the property. “”” def
__init__(self, func): functools.update_wrapper(self, func)
self.func = func def __get__(self, obj, cls): if obj is None: return
self value = obj.__dict__[self.func.__name__] = self.func(obj)
return value class TestClz(object): @cached_property def
complex_calc(self): print ‘very complex_calc’ return sum(range(100))
if __name__==’__main__’: t = TestClz() print ‘>>> first
call’ print t.complex_calc print ‘>>> second call’ print
t.complex_calc

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
import functools, time
class cached_property(object):
    """ A property that is only computed once per instance and then replaces
        itself with an ordinary attribute. Deleting the attribute resets the
        property. """
 
    def __init__(self, func):
        functools.update_wrapper(self, func)
        self.func = func
 
    def __get__(self, obj, cls):
        if obj is None: return self
        value = obj.__dict__[self.func.__name__] = self.func(obj)
        return value
 
class TestClz(object):
    @cached_property
    def complex_calc(self):
        print ‘very complex_calc’
        return sum(range(100))
 
if __name__==’__main__’:
    t = TestClz()
    print ‘>>> first call’
    print t.complex_calc
    print ‘>>> second call’
    print t.complex_calc

cached_property是一个non-data
descriptor。在TestClz中,用cached_property装饰方法complex_calc,返回值是一个descriptor实例,所以在调用的时候没有使用小括号。

第一次调用t.complex_calc之前,obj(t)的__dict__中没有”complex_calc“,
根据查找顺序第三条,执行cached_property.__get__,
这个函数代用缓存的complex_calc函数计算出结果,并且把结果放入obj.__dict__。那么第二次访问t.complex_calc的时候,根据查找顺序,第二条有限于第三条,所以就直接返回obj.__dict__[‘complex_calc’]。bottle的源码中还有两个descriptor,非常厉害!

 

类属性查找

前面提到过,类的也是对象,类是元类(metaclass)的实例,所以类属性的查找顺序基本同上。区别在于第二步,由于Clz可能有基类,所以是在Clz及其基类的__dict__”查找“attr,注意这里的查找并不是直接返回clz.__dict__[‘attr’]。具体来说,这第二步分为以下两种情况:

(2.1)如果clz.__dict__[‘attr’]是一个descriptor(不管是data
descriptor还是non-data descriptor),都调用其__get__方法

(2.2)否则返回clz.__dict__[‘attr’]

这就解释了一个很有意思的问题:method与function的问题

Python

>>> class Widget(object): … def func(self): … pass …
>>> w = Widget() >>> Widget.__dict__
dict_proxy({‘__dict__’: <attribute ‘__dict__’ of ‘Widget’
objects>, ‘__module__’: ‘__main__’, ‘__weakref__’:
<attribute ‘__weakref__’ of ‘Widget’ objects>, ‘__doc__’:
None, ‘func’: <function func at 0x7fdc7d0d1668>}) >>>
w.__dict__ {} >>> Widget.__dict__[‘func’]
<function func at 0x7fdc7d0d1668> >>> Widget.func
<unbound method Widget.func> >>>

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
>>> class Widget(object):
… def func(self):
… pass
>>> w = Widget()
>>> Widget.__dict__
dict_proxy({‘__dict__’: <attribute ‘__dict__’ of ‘Widget’ objects>, ‘__module__’: ‘__main__’, ‘__weakref__’: <attribute ‘__weakref__’ of ‘Widget’ objects>, ‘__doc__’: None, ‘func’: <function func at 0x7fdc7d0d1668>})
>>> w.__dict__
{}
 
 
>>> Widget.__dict__[‘func’]
<function func at 0x7fdc7d0d1668>
>>> Widget.func
<unbound method Widget.func>
>>>

Widget是一个之定义了一个func函数的类,func是类的属性,这个也可以通过Widget.__dict__、w.__dict__看到。Widget.__dict__[‘func’]返回的是一个function,但Widget.func是一个unbound
method,即Widget.func并不等同于Widget.__dict__[‘func’],按照前面的类属性的访问顺序,我们可以怀疑,func是一个descriptor,这样才不会走到第2.2这种情况。验证如下:

Python

>>> dir(Widget.__dict__[‘func’]) [‘__call__’,
‘__class__’, ‘__closure__’, ‘__code__’, ‘__defaults__’,
‘__delattr__’, ‘__dict__’, ‘__doc__’, ‘__format__’,
‘__get__’, ‘__getattribute__’, ‘__globals__’,
‘__hash__’, ‘__init__’, ‘__module__’, ‘__name__’,
‘__new__’, ‘__reduce__’, ‘__reduce_ex__’, ‘__repr__’,
‘__setattr__’, ‘__sizeof__’, ‘__str__’,
‘__subclasshook__’, ‘func_closure’, ‘func_code’, ‘func_defaults’,
‘func_dict’, ‘func_doc’, ‘func_globals’, ‘func_name’]

1
2
>>> dir(Widget.__dict__[‘func’])
[‘__call__’, ‘__class__’, ‘__closure__’, ‘__code__’, ‘__defaults__’, ‘__delattr__’, ‘__dict__’, ‘__doc__’, ‘__format__’, ‘__get__’, ‘__getattribute__’, ‘__globals__’, ‘__hash__’, ‘__init__’, ‘__module__’, ‘__name__’, ‘__new__’, ‘__reduce__’, ‘__reduce_ex__’, ‘__repr__’, ‘__setattr__’, ‘__sizeof__’, ‘__str__’, ‘__subclasshook__’, ‘func_closure’, ‘func_code’, ‘func_defaults’, ‘func_dict’, ‘func_doc’, ‘func_globals’, ‘func_name’]

元类概念回顾(>>传送门点这里<<)

属性赋值

Python的属性赋值(attribute assignment)也会受到descriptor(data
descriptor)的影响,同时也会受到__setattr__函数的影响。当然Python中还有一个setattr,setattr(x,
‘foobar’, 123)等价于x.foobar = 123,二者都叫attribute assignment。

首先看看__setattr__:

object.__setattr__(self, name, value)
Called when an attribute assignment is attempted. This is called
instead of the normal mechanism

那什么是normal mechanism,简单来说就是x.__dict__[‘foobar’] =
123,不管’foobar’之前是否是x的属性(当然赋值之后就一定是了)。但是如果‘’foobar‘’是类属性,且是data
descriptor,那么回优先调用__set__。我们来看一个例子:

class MaxValDes(object): def __init__(self, attr, max_val):
self.attr = attr self.max_val = max_val def __get__(self,
instance, typ): return instance.__dict__[self.attr] def
__set__(self, instance, value): instance.__dict__[self.attr] =
min(self.max_val, value) print ‘MaxValDes __set__’, self.attr,
instance.__dict__[self.attr] class Widget(object): a =
MaxValDes(‘a’, 10) def __init__(self): self.a = 0 # def
__setattr__(self, name, value): # self.__dict__[name] = value
# print ‘Widget __setattr__’, name, self.__dict__[name] if
__name__ == ‘__main__’: w0 = Widget() w0.a = 123

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
class MaxValDes(object):
    def __init__(self, attr, max_val):
        self.attr = attr
        self.max_val = max_val
 
    def __get__(self, instance, typ):
        return instance.__dict__[self.attr]
 
    def __set__(self, instance, value):
        instance.__dict__[self.attr] = min(self.max_val, value)
        print ‘MaxValDes __set__’, self.attr, instance.__dict__[self.attr]
 
class Widget(object):
    a = MaxValDes(‘a’, 10)
    def __init__(self):
        self.a = 0
 
    # def __setattr__(self, name, value):
    #     self.__dict__[name] = value
    #     print ‘Widget __setattr__’, name, self.__dict__[name]
 
if __name__ == ‘__main__’:
    w0 = Widget()
    w0.a = 123

输出如下:

Python

MaxValDes __set__ a 0 MaxValDes __set__ a 10

1
2
MaxValDes __set__ a 0
MaxValDes __set__ a 10

可以看到,即使Widget的实例也有一个‘a’属性,但是调用w.a的时候会调用类属性‘a’(一个descriptor)的__set__方法。如果不注释掉第18到第20行,输出如下

Python

Widget __setattr__ a 0 Widget __setattr__ a 123

1
2
Widget __setattr__ a 0
Widget __setattr__ a 123

可以看到,优先调用Widget 的__setattr__方法。因此:对于属性赋值,obj
= Clz(), 那么obj.attr = var,按照这样的顺序:

  1. 如果Clz定义了__setattr__方法,那么调用该方法,否则
  2. 如果“attr”是出现在Clz或其基类的__dict__中, 且attr是data
    descriptor, 那么调用其__set__方法, 否则
  3. 等价调用obj.__dict__[‘attr’] = var

图片 3图片 4

references

  • Descriptor HowTo Guide,
  • Object attribute lookup in Python,
  • python __set__ __get__ 等解释,

    1 赞 3 收藏 1
    评论

#!/usr/bin/env python
# -*- coding: utf-8 -*-
# @Time    : 2018/05/17 8:25
# @Author  : MJay_Lee
# @File    : 列表推导式.py
# @Contact : limengjiejj@hotmail.com
egg_list = []
for i in range(10):
    egg_list.append('egg%s' % i)

print(egg_list)
# ['egg0', 'egg1', 'egg2', 'egg3', 'egg4', 'egg5', 'egg6', 'egg7', 'egg8', 'egg9']

egg_list2 = ['egg%s' % i for i in range(10)]
print(egg_list2)

列表推导式

图片 5图片 6

class Foo:
    x=1
    def __init__(self,y):
        self.y=y

    def __getattr__(self, item):
        print('----> from getattr:你找的属性不存在')


    def __setattr__(self, key, value):
        print('----> from setattr')
        # self.key=value #这就无限递归了,你好好想想
        # self.__dict__[key]=value #应该使用它

    def __delattr__(self, item):
        print('----> from delattr')
        # del self.item #无限递归了
        self.__dict__.pop(item)

#__setattr__添加/修改属性会触发它的执行
f1=Foo(10)
print(f1.__dict__) # 因为你重写了__setattr__,凡是赋值操作都会触发它的运行,你啥都没写,就是根本没赋值,除非你直接操作属性字典,否则永远无法赋值
f1.z=3
print(f1.__dict__)

#__delattr__删除属性的时候会触发
f1.__dict__['a']=3#我们可以直接修改属性字典,来完成添加/修改属性的操作
del f1.a
print(f1.__dict__)

#__getattr__只有在使用点调用属性且属性不存在的时候才会触发
f1.xxxxxx

__getattr__,__setattr__

补充:

图片 7图片 8

class Foo:
    def __init__(self,x):
        self.x=x

    def __getattr__(self, item):
        print('执行的是我')
        # return self.__dict__[item]
    def __getattribute__(self, item):
        print('不管是否存在,我都会执行')
        raise AttributeError('哈哈')

f1=Foo(10)
f1.x
f1.xxxxxx

#当__getattribute__与__getattr__同时存在,只会执行__getattrbute__,除非__getattribute__在执行过程中抛出异常AttributeError

区分__getattr__,__getattribute__

 反射(update和save两个功能代码里,拼接SQL语句时,给参数赋值时时需要用上): 

图片 9图片 10

def getattr(object, name, default=None): # known special case of getattr
    """
    getattr(object, name[, default]) -> value

    Get a named attribute from an object; getattr(x, 'y') is equivalent to x.y.
    When a default argument is given, it is returned when the attribute doesn't
    exist; without it, an exception is raised in that case.
    """
    pass


def setattr(x, y, v): # real signature unknown; restored from __doc__
    """
    Sets the named attribute on the given object to the specified value.

    setattr(x, 'y', v) is equivalent to ``x.y = v''
    """
    pass


def delattr(x, y): # real signature unknown; restored from __doc__
    """
    Deletes the named attribute from the given object.

    delattr(x, 'y') is equivalent to ``del x.y''
    """
    pass

getattr及其它相关属性

图片 11图片 12

class BlackMedium:
    feature='Ugly'
    def __init__(self,name,addr):
        self.name=name
        self.addr=addr

    def sell_house(self):
        print('%s 黑中介卖房子啦,傻逼才买呢,但是谁能证明自己不傻逼' %self.name)
    def rent_house(self):
        print('%s 黑中介租房子啦,傻逼才租呢' %self.name)

b1=BlackMedium('万成置地','回龙观天露园')

#检测是否含有某属性
print(hasattr(b1,'name'))
print(hasattr(b1,'sell_house'))

#获取属性
n=getattr(b1,'name')
print(n)
func=getattr(b1,'rent_house')
func()

# getattr(b1,'aaaaaaaa') #报错
print(getattr(b1,'aaaaaaaa','不存在啊'))

#设置属性
setattr(b1,'sb',True)
setattr(b1,'show_name',lambda self:self.name+'sb')
print(b1.__dict__)
print(b1.show_name(b1))

#删除属性
delattr(b1,'addr')
delattr(b1,'show_name')
delattr(b1,'show_name111')#不存在,则报错

print(b1.__dict__)

类与对象的四个操作属性示例

操作类与对象的属性的补充:

图片 13图片 14

class Foo:

    def __del__(self):
        print('执行我啦')

f1=Foo()
del f1
print('------->')

#输出结果
执行我啦
------->

----------------------以下是另一种情况
class Foo:

    def __del__(self):
        print('执行我啦')

f1=Foo()
# del f1
print('------->')

#输出结果
------->
执行我啦

典型的应用场景:

创建数据库类,用该类实例化出数据库链接对象,对象本身是存放于用户空间内存中,而链接则是由操作系统管理的,存放于内核空间内存中

当程序结束时,python只会回收自己的内存空间,即用户态内存,而操作系统的资源则没有被回收,这就需要我们定制__del__,在对象被删除前向操作系统发起关闭数据库链接的系统调用,回收资源

析构函数,__del__方法

图片 15图片 16

format_dict={
    'nat':'{obj.name}-{obj.addr}-{obj.type}',#学校名-学校地址-学校类型
    'tna':'{obj.type}:{obj.name}:{obj.addr}',#学校类型:学校名:学校地址
    'tan':'{obj.type}/{obj.addr}/{obj.name}',#学校类型/学校地址/学校名
}
class School:
    def __init__(self,name,addr,type):
        self.name=name
        self.addr=addr
        self.type=type

    def __repr__(self):
        return 'School(%s,%s)' %(self.name,self.addr)
    def __str__(self):
        return '(%s,%s)' %(self.name,self.addr)

    def __format__(self, format_spec):
        # if format_spec
        if not format_spec or format_spec not in format_dict:
            format_spec='nat'
        fmt=format_dict[format_spec]
        return fmt.format(obj=self)

s1=School('oldboy1','北京','私立')
print('from repr: ',repr(s1))
print('from str: ',str(s1))
print(s1)

'''
str函数或者print函数--->obj.__str__()
repr或者交互式解释器--->obj.__repr__()
如果__str__没有被定义,那么就会使用__repr__来代替输出
注意:这俩方法的返回值必须是字符串,否则抛出异常
'''
print(format(s1,'nat'))
print(format(s1,'tna'))
print(format(s1,'tan'))
print(format(s1,'asfdasdffd'))

自定义打印格式,__str__方法

单例 (>>传送门点这里<<)

图片 17图片 18

#!/usr/bin/env python
# -*- coding: utf-8 -*-
# @Time    : 2018/04/18 17:47
# @Author  : MJay_Lee
# @File    : 单例.py
# @Contact : limengjiejj@hotmail.com

# 基于元类实现单例模式
# 单例:即单个实例,指的是同一个类实例化多次的结果指向同一个对象,用于节省空间(场景:假若从配置文件中读取配置来进行实例化,在配置相同的情况下,就没必要重复产生对象浪费内存了)



# 方式一:定义一个类方法实现单例模式
# import setting
#
# class Mysql:
#     instance = None
#     def __init__(self,host,port):
#         self.host = host
#         self.port = port
#
#     @classmethod
#     def from_conf(self):
#         if not Mysql.instance:
#             res = Mysql(setting.HOST, setting.PORT)
#             Mysql.instance = res
#         return Mysql.instance
#
# # con1 = Mysql('127.0.0.1',80) # <__main__.Mysql object at 0x000000A9F7FC7978>
# # con2 = Mysql('127.0.0.1',80) # <__main__.Mysql object at 0x000000A9F7FD8710>
# # con3 = Mysql('127.0.0.1',80) # <__main__.Mysql object at 0x000000A9F7E09C88>
# # print(con1,con2,con3)
# #
#
# # con1 = Mysql.from_conf() # <__main__.Mysql object at 0x000000BB72BA4DD8>
# # con2 = Mysql.from_conf() # <__main__.Mysql object at 0x000000BB72BA4E48>
# # con3 = Mysql.from_conf() # <__main__.Mysql object at 0x000000BB72BA4E80>
# # print(con1,con2,con3)
#
# con1 = Mysql.from_conf() # <__main__.Mysql object at 0x000000BD5BBA4DD8>
# con2 = Mysql.from_conf() # <__main__.Mysql object at 0x000000BD5BBA4DD8>
# con3 = Mysql.from_conf() # <__main__.Mysql object at 0x000000BD5BBA4DD8>
# print(con1 is con2 is con3) # True


# # 方式二:定制元类实现
# # 若从配置文件取相同配置产生对象则实现单例,若传值则新建对象
import setting

class Mymeta(type):
    def __init__(self,name,bases,dic): # 定义类Mysql时就触发

        # 事先从配置文件中取配置来造一个Mysql的实例出来
        self.__instance = object.__new__(self) # 产生对象
        self.__init__(self.__instance,setting.HOST,setting.PORT) # 初始化对象
        #上述两步可合并下面一步
        # self.__instance = super().__call__(*args,**kwargs)

        super().__init__(name,bases,dic)

    def __call__(self, *args, **kwargs): # Mysql(...)时触发
        if args or kwargs: # Mymeta类的对象括号内传值则新建obj,否则返回self.__instance
            obj = object.__new__(self)
            self.__init__(obj,*args,**kwargs)
            return obj
        return self.__instance


# Mysql = Mymeta('Mysql',(obj,),class_dic)
class Mysql(metaclass=Mymeta):
    def __init__(self,host,port):
        self.host = host
        self.port = port

con1 = Mysql()
con2 = Mysql()
# con3 = Mysql() # <__main__.Mysql object at 0x0000008BA7E24DD8>
# con4 = Mysql('127.0.0.1',80) # <__main__.Mysql object at 0x0000004B4B904EF0>,若Mymeta类的对象(Mysql)括号内传值则新建obj
# print(con4) # True


# 装饰器实现单例
# import setting
#
# def single_obj(cls):
#     __instance = cls(setting.HOST,setting.PORT)
#     def wrapper(*args, **kwargs):
#         if args or kwargs:
#             obj = cls(*args, **kwargs)
#             return obj
#         return __instance
#     return wrapper
#
# @single_obj
# class Mysql:
#     def __init__(self,host,port):
#         self.host = host
#         self.port = port
#
# con1 = Mysql() # <__main__.Mysql object at 0x0000001F978D9C88>
# con2 = Mysql() # <__main__.Mysql object at 0x0000001F978D9C88>
# con3 = Mysql('127.0.0.1',80) # <__main__.Mysql object at 0x0000001F98AE4DD8>
#
# print(con1 is con2) # True

三个方法实现单例–示例

 


 

ORM简介

ORM即Object Relational Mapping,全称对象关系映射
当我们需要对数据库进行操作时,势必需要通过连接数据、调用sql语句、执行sql语句等操作,ORM将数据库中的表,字段,行与我们面向对象编程的类及其方法,属性等一一对应,即将该部分操作封装起来,程序猿不需懂得sql语句即可完成对数据库的操作。

 图片 19

一、知识储备:

1、在实例化一个user对象的时候,可以user=User(name=’lqz’,password=’123’)

2 也可以 user=User()

    user[‘name’]=’lqz’
    user[‘password’]=’123′
3 也可以 user=User()

    user.name=’lqz’
    user.password=’password’

前两种,可以通过继承字典dict来实现,第三种,用getattr和setattr:

*__getattr__
拦截点号运算
。当对未定义的属性名称和实例进行点号运算时,就会用属性名作为字符串调用这个方法。如果继承树可以找到该属性,则不调用此方法*

*__setattr__会拦截所有属性的的赋值语句。如果定义了这个方法,self.arrt
= value 就会变成self,__setattr__(“attr”,
value).这个需要注意。当在__setattr__方法内对属性进行赋值是,不可使用self.attr
= value,因为他会再次调用self,__setattr__(“attr”,
value),则会形成无穷递归循环,最后导致堆栈溢出异常。应该通过对属性字典做索引运算来赋值任何实例属性,也就是使用self.__dict__[‘name’]
= value*

 二、定义Model基类

# 在ModelsMetaclass中自定义拦截实例化对象的方法
class Models(dict,metaclass=ModelsMetaclass):
    # k,v形式的值
    def __init__(self,**kwargs):
        super().__init__(**kwargs)

    # 写存
    def __setattr__(self, key, value):
        self[key] = value

    # 读取
    def __getattr__(self, item):
        try:
            return self[item]
        except KeyError:
            raise ('没有该属性')

三、定义Field

数据库中每一列数据,都有:列名,列的数据类型,是否是主键,默认值

# 表示一个列:列名,列的类型,列的主键和默认值
class Field:
    def __init__(self,name,column_type,primary_key,default):
        self.name = name
        self.column_type = column_type
        self.primary_key = primary_key
        self.default = default


class StringField(Field):
    def __init__(self,name=None,column_type='varchar(200)',primary_key=False,default=None):
        super().__init__(name,column_type,primary_key,default)


class IntegerField(Field):
    def __init__(self,name=None,column_type='int',primary_key=False,default=None):
        super().__init__(name,column_type,primary_key,default)

四、定义元类

数据库中的每个表,都有表名,每一列的列名,以及主键是哪一列

既然我要用数据库中的表,对应这一个程序中的类,那么我这个类也应该有这些类属性

但是不同的类这些类属性又不尽相同,所以我应该怎么做?在元类里拦截类的创建过程,然后把这些东西取出来,放到类里面

class ModelsMetaclass(type):
    def __new__(cls,name,bases,attrs):

        if name == 'Models': #
            return type.__new__(cls, name, bases, attrs)
        table_name = attrs.get('table_name', None) #字典取值,中括号或.get
        if not table_name:
            table_name = name

        primary_key = None
        mappings = dict()
        for k, v in attrs.items():
            if isinstance(v, Field):  # v 是不是Field的对象
                mappings[k] = v
                if v.primary_key: # v是基类对象,即判断该字段的主键

                    # 找到主键
                    if primary_key:
                        raise TypeError('主键重复:%s' % k)
                    primary_key = k

        for k in mappings.keys():
            attrs.pop(k) # 执行完此步后,attrs中只剩余有__属性__
        if not primary_key:
            raise TypeError('没有主键')
        attrs['table_name'] = table_name
        attrs['primary_key'] = primary_key
        attrs['mappings'] = mappings
        return type.__new__(cls, name, bases, attrs)

五、基于pymysql的数据库操作类(单例)

#!/usr/bin/env python
# -*- coding: utf-8 -*-
# @Time    : 2018/05/15 10:44
# @Author  : MJay_Lee
# @File    : mysql_singleton.py
# @Contact : limengjiejj@hotmail.com

import pymysql

class Mysql_interface:
    __instense = None
    def __init__(self):
        self.conn = pymysql.connect(
            host = '127.0.0.1',
            port = 3306,
            user = 'root',
            password = '123456',
            charset = 'utf8',
            database = 'youku',
            autocommit = True
        )
        self.cursor = self.conn.cursor(cursor=pymysql.cursors.DictCursor)

    def close_db(self):
        self.cursor.close()
        self.conn.close()

    def select(self,sql,args):
        self.cursor.execute(sql,args)
        re = self.cursor.fetchall()

        return re

    def execute(self,sql,args):
        try:
            self.cursor.execute(sql,args)
            affected = self.cursor.rowcount
        except BaseException as e:
            print(e)
        return affected

    @classmethod
    def singleton(cls):
        if not cls.__instense:
            cls.__instense = cls()
        return cls.__instense



if __name__ == '__main__':
    ms = Mysql_interface()
    re = ms.select('select * from user where id = %s',1)
    print(re)

六、继续Models基类

Models类是所有要对应数据库表类的基类,所以,Models的元类应该是咱们上面写的那个

而每个数据库表对应类的对象,都应该有查询、插入、保存,方法

所以:

# 在ModelsMetaclass中自定义拦截实例化对象的方法
class Models(dict,metaclass=ModelsMetaclass):
    # k,v形式的值
    def __init__(self,**kwargs):
        super().__init__(**kwargs)

    # 写存
    def __setattr__(self, key, value):
        self[key] = value

    # 读取
    def __getattr__(self, item):
        try:
            return self[item]
        except KeyError:
            raise ('没有该属性')

    @classmethod
    def select_one(cls,**kwargs):
        '''
        查一条
        :param kwargs:
        :return:
        '''
        key = list(kwargs.keys())[0]
        value = kwargs[key]

        # select * from user where id=%s
        sql = 'select * from %s where %s =?' % (cls.table_name,key)
        sql = sql.replace('?','%s')
        ms = mysql_singleton.Mysql_interface().singleton()
        re = ms.select(sql,value) # 得到re字典对象
        if re:
            # attrs = {'name':'lmj','password':123}
            # User(**attrs)
            # 相当于 User(name='lmj',password=123)
            return cls(**re[0])
        else:
            return

    @classmethod
    def select_many(cls, **kwargs):
        '''
        查多条
        :param kwargs:
        :return:
        '''
        ms = mysql_singleton.Mysql_interface().singleton()
        if kwargs:
            key = list(kwargs.keys())[0]
            value = kwargs[key]

            sql = 'select * from %s where %s =?' % (cls.table_name, key)
            sql = sql.replace('?', '%s')
            re = ms.select(sql, value)  # 得到re字典对象
        else:
            sql = 'select * from %s' % (cls.table_name)
            re = ms.select(sql)
        if re:
            obj_list = [cls(**r) for r in re]
            return obj_list
        else:
            return


    def update(self):
        ms = mysql_singleton.Mysql_interface().singleton()
        # update user set name = ?,password = ? where id = ?

        filed_data = [] # name = ?,password = ?
        pr = None
        args = [] # 字段的值
        for k,v in self.mappings.items():
            if v.primary_key:
                pr = getattr(self,v.name,v.default)
            else:
                filed_data.append(v.name + '=?')
                args.append(getattr(self,v.name,v.default))

        sql = 'update %s set %s where %s = %s' % (self.table_name,','.join(filed_data),self.primary_key,pr)
        sql = sql.replace('?','%s')
        ms.execute(sql,args)


    def save(self):
        ms = mysql_singleton.Mysql_interface().singleton()
        # insert into user(name,password) values (?,?)
        field_data = []
        args = []
        value_data = []
        for k,v in self.mappings.items():
            if not v.primary_key:
                field_data.append(v.name)
                args.append(getattr(self,v.name,v.default))
                value_data.append('?')

        sql = 'insert into %s(%s) VALUES (%s)' % (self.table_name,','.join(field_data),','.join(value_data))
        sql = sql.replace('?','%s')
        ms.execute(sql,args)

 数据库池版,orm_pool的配置:

图片 20图片 21

from DBUtils.PooledDB import PooledDB
import pymysql

POOL = PooledDB(
    creator=pymysql,  # 使用链接数据库的模块
    maxconnections=6,  # 连接池允许的最大连接数,0和None表示不限制连接数
    mincached=2,  # 初始化时,链接池中至少创建的空闲的链接,0表示不创建
    maxcached=5,  # 链接池中最多闲置的链接,0和None不限制
    maxshared=3,  # 链接池中最多共享的链接数量,0和None表示全部共享。PS: 无用,因为pymysql和MySQLdb等模块的 threadsafety都为1,所有值无论设置为多少,_maxcached永远为0,所以永远是所有链接都共享。
    blocking=True,  # 连接池中如果没有可用连接后,是否阻塞等待。True,等待;False,不等待然后报错
    maxusage=None,  # 一个链接最多被重复使用的次数,None表示无限制
    setsession=[],  # 开始会话前执行的命令列表。
    ping=0,
    # ping MySQL服务端,检查是否服务可用。
    host='127.0.0.1',
    port=3306,
    user='root',
    password='123456',
    database='youku',
    charset='utf8',
    autocommit = True
)

def func():
    # 检测当前正在运行连接数的是否小于最大链接数,如果不小于则:等待或报raise TooManyConnections异常
    # 否则
    # 则优先去初始化时创建的链接中获取链接 SteadyDBConnection。
    # 然后将SteadyDBConnection对象封装到PooledDedicatedDBConnection中并返回。
    # 如果最开始创建的链接没有链接,则去创建一个SteadyDBConnection对象,再封装到PooledDedicatedDBConnection中并返回。
    # 一旦关闭链接后,连接就返回到连接池让后续线程继续使用。
    conn = POOL.connection()

    # print('链接被拿走了', conn._con)
    # print('池子里目前有', POOL._idle_cache, 'rn')

    cursor = conn.cursor()
    cursor.execute('select * from user')
    result = cursor.fetchall()
    print(result)
    conn.close()

if __name__ == '__main__':

    func()

orm_pool.py

mysql_pool的配置:

图片 22图片 23

#!/usr/bin/env python
# -*- coding: utf-8 -*-
# @Time    : 2018/05/15 10:44
# @Author  : MJay_Lee
# @File    : mysql_pool.py
# @Contact : limengjiejj@hotmail.com

from video_web_mysql.orm_pool import orm_pool
import pymysql

class Mysql_interface:
    def __init__(self):
        self.conn = orm_pool.POOL.connection()
        self.cursor = self.conn.cursor(cursor=pymysql.cursors.DictCursor)

    def close_db(self):
        self.cursor.close()
        self.conn.close()

    def select(self,sql,args=None):
        self.cursor.execute(sql,args)
        re = self.cursor.fetchall()

        return re

    def execute(self,sql,args):
        try:
            self.cursor.execute(sql,args)
            affected = self.cursor.rowcount
        except BaseException as e:
            print(e)
        return affected


if __name__ == '__main__':
    ms = Mysql_interface()
    re = ms.select('select * from user where id = %s',1)
    print(re)

mysql_pool.py

fuckorm的完整源码:

图片 24图片 25

#!/usr/bin/env python
# -*- coding: utf-8 -*-
# @Time    : 2018/05/15 8:58
# @Author  : MJay_Lee
# @File    : fuckorm.py
# @Contact : limengjiejj@hotmail.com

from video_web_mysql.orm_pool import mysql_pool


# 表示一个列:列名,列的类型,列的主键和默认值
class Field:
    def __init__(self,name,column_type,primary_key,default):
        self.name = name
        self.column_type = column_type
        self.primary_key = primary_key
        self.default = default


class StringField(Field):
    def __init__(self,name=None,column_type='varchar(200)',primary_key=False,default=None):
        super().__init__(name,column_type,primary_key,default)


class IntegerField(Field):
    def __init__(self,name=None,column_type='int',primary_key=False,default=None):
        super().__init__(name,column_type,primary_key,default)


class ModelsMetaclass(type):
    def __new__(cls,name,bases,attrs):

        if name == 'Models': #
            return type.__new__(cls, name, bases, attrs)
        table_name = attrs.get('table_name', None) #字典取值,中括号或.get
        if not table_name:
            table_name = name

        primary_key = None
        mappings = dict()
        for k, v in attrs.items():
            if isinstance(v, Field):  # v 是不是Field的对象
                mappings[k] = v
                if v.primary_key: # v是基类对象,即判断该字段的主键

                    # 找到主键
                    if primary_key:
                        raise TypeError('主键重复:%s' % k)
                    primary_key = k

        for k in mappings.keys():
            attrs.pop(k) # 执行完此步后,attrs中只剩余有__属性__
        if not primary_key:
            raise TypeError('没有主键')
        attrs['table_name'] = table_name
        attrs['primary_key'] = primary_key
        attrs['mappings'] = mappings
        return type.__new__(cls, name, bases, attrs)


# 在ModelsMetaclass中自定义拦截实例化对象的方法
class Models(dict,metaclass=ModelsMetaclass):
    # k,v形式的值
    def __init__(self,**kwargs):
        super().__init__(**kwargs)

    # 写存
    def __setattr__(self, key, value):
        self[key] = value

    # 读取
    def __getattr__(self, item):
        try:
            return self[item]
        except KeyError:
            raise ('没有该属性')

    @classmethod
    def select_one(cls,**kwargs):
        '''
        查一条
        :param kwargs:
        :return:
        '''
        key = list(kwargs.keys())[0]
        value = kwargs[key]

        # select * from user where id=%s
        sql = 'select * from %s where %s =?' % (cls.table_name,key)
        sql = sql.replace('?','%s')
        ms = mysql_pool.Mysql_interface()
        re = ms.select(sql,value) # 得到re字典对象
        if re:
            # attrs = {'name':'lmj','password':123}
            # User(**attrs)
            # 相当于 User(name='lmj',password=123)
            return cls(**re[0])
        else:
            return

    @classmethod
    def select_many(cls, **kwargs):
        '''
        查多条
        :param kwargs:
        :return:
        '''
        ms = mysql_pool.Mysql_interface()
        if kwargs:
            key = list(kwargs.keys())[0]
            value = kwargs[key]

            sql = 'select * from %s where %s =?' % (cls.table_name, key)
            sql = sql.replace('?', '%s')
            re = ms.select(sql, value)  # 得到re字典对象
        else:
            sql = 'select * from %s' % (cls.table_name)
            re = ms.select(sql)
        if re:
            obj_list = [cls(**r) for r in re]
            return obj_list
        else:
            return


    def update(self):
        ms = mysql_pool.Mysql_interface()
        # update user set name = ?,password = ? where id = ?

        filed_data = [] # name = ?,password = ?
        pr = None
        args = [] # 字段的值
        for k,v in self.mappings.items():
            if v.primary_key:
                pr = getattr(self,v.name,v.default)
            else:
                filed_data.append(v.name + '=?')
                args.append(getattr(self,v.name,v.default))

        sql = 'update %s set %s where %s = %s' % (self.table_name,','.join(filed_data),self.primary_key,pr)
        sql = sql.replace('?','%s')
        ms.execute(sql,args)


    def save(self):
        ms = mysql_pool.Mysql_interface()
        # insert into user(name,password) values (?,?)
        field_data = []
        args = []
        value_data = []
        for k,v in self.mappings.items():
            # 此处判断是否为自增主键,否则插入时避免还需手动输入主键ID
            if not v.primary_key:
                field_data.append(v.name)
                value_data.append('?')
                args.append(getattr(self, v.name, v.default))

        sql = 'insert into %s(%s) VALUES (%s)' % (self.table_name,','.join(field_data),','.join(value_data))
        sql = sql.replace('?','%s')
        ms.execute(sql,args)


class User(Models):
    '''
    首先赋值表名
    其次根据数据库表结构来赋值
    '''
    table_name = 'user'
    # k   v(Field的对象)
    id = IntegerField('id',primary_key=True)
    password = StringField('password')

class Notice(Models):
    table_name = 'notice'
    id = IntegerField('id',primary_key=True)
    name = StringField('name')
    content = StringField('content')
    user_id = IntegerField('user_id')


if __name__ == '__main__':
    # notice = Notice.select_one(id=1)
    # print(notice.content)

    # notice_list = Notice.select_many(id=1)
    # print(notice_list)

    # notice.name = '改变了'
    # notice.update()

    notice = Notice(name='123',content='新插入',user_id=1)
    notice.save()

fuckorm.py

增删改查,基础功能均亲测有效。

Leave a Comment.