Python数据载入代码

Python中用于数据载入的代码通常取决于你要读取的数据类型和文件格式。以下是一些常见的数据载入方法：

1. **CSV文件** (使用`pandas`库):

import pandas as pd
data = pd.read_csv('filename.csv')

2. **Excel文件** (同样用pandas):

data = pd.read_excel('filename.xlsx')

3. **JSON文件** (pandas或`json`模块):

import json
with open('filename.json') as f:
    data = json.load(f)

4. **文本文件** (逐行读取):

with open('filename.txt', 'r') as file:
    data = [line.strip() for line in file]

5. **SQL数据库** (使用`sqlite3`, `psycopg2`等库):

import sqlite3
conn = sqlite3.connect('database.db')
cursor = conn.cursor()
data = cursor.execute("SELECT * FROM table_name").fetchall()

6. **CSV文件（异步加载）** (使用`dask`库):

import dask.dataframe as dd
data = dd.read_csv('filename.csv')

7. **HDF5文件** (h5py库):

import h5py
with h5py.File('filename.hdf5', 'r') as f:
    data = f['dataset_name']

8. **CSV文件（Pandas DataFrame分块读取大型文件）**:

chunksize = 10 ** 6  # 分块大小
chunks = []
for chunk in pd.read_csv('large_file.csv', chunksize=chunksize):
    chunks.append(chunk)
data = pd.concat(chunks)