[Python知识库]Python、pandas读取和存储json、xlsx、csv文件速度比较

利用原有装饰器函数 @timeit_test(number=3, repeat=3) 进行测试，源代码如下：

    @timeit_test(number=3, repeat=3)
    def t1():
        with open(r'.\stock\stock_pool.json', 'r', encoding='utf-8') as f:
            stock_dict = json.load(f)
        # print(stock_dict['华银电力'], stock_dict['商络电子'])

    with open(r'.\stock\stock_pool.json', 'r', encoding='utf-8') as f:
        stock_dict = json.load(f)        
    @timeit_test(number=3, repeat=3)
    def t2():
        with open(r'.\stock\stock_pool.json', 'w', encoding='utf-8') as f:
            json.dump(stock_dict, f, ensure_ascii=False, indent=4)


    @timeit_test(number=3, repeat=3)
    def t3():
        df = pd.read_excel(r'.\stock\stock_all_code_and_name.xlsx')


    df = pd.read_excel(r'.\stock\stock_all_code_and_name.xlsx')
    @timeit_test(number=3, repeat=3)
    def t4():
        writer = pd.ExcelWriter(r'C:\Users\Administrator\pythonProject\Stock_API\stock\stock_all_code_and_name.xlsx')
        df.to_excel(writer, float_format='%.2f')
        writer.save()


    @timeit_test(number=3, repeat=3)
    def t5():
        df.to_csv(r'.\stock\stock_csv.csv', columns=df.columns, index=True)

    @timeit_test(number=3, repeat=3)
    def t6():
        df_csvload = pd.read_csv(r'.\stock\stock_csv.csv', parse_dates=True, index_col=0, encoding='utf-8')

    print('json read t1:')
    t1()
    print('json write t2:')
    t2()
    print('excel read t3:')
    t3()
    print('excel write t4:')
    t4()
    print('csv write t5:')
    t5()
    print('csv read t6:')
    t6()

测试输出结果如下：

json read t1:
Time used: 0.01951 s.
Time used: 0.01962 s.
Time used: 0.01987 s.
json write t2:
Time used: 0.06542 s.
Time used: 0.06272 s.
Time used: 0.06836 s.
excel read t3:
Time used: 5.48027 s.
Time used: 5.41701 s.
Time used: 5.87145 s.
excel write t4:
Time used: 11.86320 s.
Time used: 11.92886 s.
Time used: 11.84569 s.
csv write t5:
Time used: 0.20016 s.
Time used: 0.18753 s.
Time used: 0.18681 s.
csv read t6:
Time used: 0.14664 s.
Time used: 0.13597 s.
Time used: 0.13688 s.

Process finished with exit code 0

测试结果表明：json 和 csv 格式文件读写，均在 0.1 左右完成；而 excel 文件读用时5秒多，写用时11秒多，比 json、csv 文件读写慢50-100倍。所以，尽量减少excel的读写，必要时采用文本格式json、csv文件进行存储信息，以提高程序运行效率。