如何将MySQL时间戳(6)读入熊猫？

我有一个MySql表,其时间戳具有微秒分辨率：

+----------------------------+------+
| time                       | seq  | 
+----------------------------+------+
| 2015-06-19 02:17:57.389509 |    0 | 
| 2015-06-19 02:17:57.934171 |   10 |
+----------------------------+------+

我想把它读成一个pandas Dataframe.运用

import pandas as pd
con = get_connection()
result = pd.read_sql("SELECT * FROM MyTable;", con=con)
print result

返回NaT(不是时间)：

    time  seq 
0   NaT    0  
1   NaT   10  

如何将其读入时间戳？

解决方法:

通常,要转换时间戳,可以使用pandas.to_datetime().

>>> import pandas as pd
>>> pd.to_datetime('2015-06-19 02:17:57.389509')
Timestamp('2015-06-19 02:17:57.389509')

从docs开始,从sql读入时,您可以显式强制将列解析为日期：

pd.read_sql_table('data', engine, parse_dates=['Date'])

或者更明确地,指定格式字符串或传递给pandas.to_datetime()的参数的字典：

pd.read_sql_table('data', engine, parse_dates={'Date': '%Y-%m-%d'})

要么

pd.read_sql_table('data', engine, parse_dates={'Date': {'format': '%Y-%m-%d %H:%M:%s'}})

添加快速概念证明.注意,我正在使用sqlITE.假设您将时间戳存储为字符串：

from sqlalchemy import create_engine, Table, Column, Integer, String, MetaData
import pandas as pd

engine = create_engine('sqlite:///:memory:', echo=True)

datapoints = [{'ts': '2015-06-19 02:17:57.389509', 'seq': 0},
              {'ts':'2015-06-19 02:17:57.934171', 'seq': 10}]
Metadata = MetaData()
mydata = Table('mydata', Metadata,
    Column('ts', String),
    Column('seq', Integer),
)
Metadata.create_all(engine)
ins = mydata.insert()
conn = engine.connect()
conn.execute(ins, datapoints)

foo = pd.read_sql_table('mydata', engine, parse_dates=['ts'])
print(foo)

输出：

                           ts  seq
0  2015-06-19 02:17:57.389509    0
1  2015-06-19 02:17:57.934171   10

或者,如果您将它们存储为日期时间对象,它的工作方式相同(代码差异是我以datetime格式将数据导入数据库)：

from datetime import datetime
from sqlalchemy import create_engine, Table, Column, Integer, DateTime, MetaData
import pandas as pd

engine = create_engine('sqlite:///:memory:', echo=True)

datapoints = [{'ts': datetime.strptime('2015-06-19 02:17:57.389509', '%Y-%m-%d %H:%M:%s.%f'), 'seq': 0},
              {'ts':datetime.strptime('2015-06-19 02:17:57.934171', '%Y-%m-%d %H:%M:%s.%f'), 'seq': 10}]
Metadata = MetaData()
mydata = Table('mydata', Metadata,
    Column('ts', DateTime),
    Column('seq', Integer),
)
Metadata.create_all(engine)
ins = mydata.insert()
conn = engine.connect()
conn.execute(ins, datapoints)

foo = pd.read_sql_table('mydata', engine, parse_dates=['ts'])
print(foo)

输出相同：

                          ts  seq
0 2015-06-19 02:17:57.389509    0
1 2015-06-19 02:17:57.934171   10

希望这可以帮助.

编辑为了解决@joris的问题,sqlite将所有日期时间对象存储为字符串,但是内置适配器在获取时会自动将这些对象转换回datetime对象.扩展第二个例子：

from sqlalchemy.sql import select
s = select([mydata])
res = conn.execute(s)
row = res.fetchone()
print(type(row['ts']))

结果为< class'datetime.datetime'>