[人工智能]＜机器学习实战＞第一章的 oecd_bli = oecd_bli[oecd_bli[“INEQUALITY“]==“TOT“ 什么意思？

在《机器学习实战》这本书里第一章有个实例：联合国统计的人均GDP与国民幸福度之间的关系。书中使用matlablib做散点图。大致的代码如下：

def prepare_country_stats(oecd_bli, gdp_per_capita):
    oecd_bli = oecd_bli[oecd_bli["INEQUALITY"]=="TOT"]
    oecd_bli = oecd_bli.pivot(index="Country", columns="Indicator", values="Value")
    gdp_per_capita.rename(columns={"2015": "GDP per capita"}, inplace=True)
    gdp_per_capita.set_index("Country", inplace=True)
    full_country_stats = pd.merge(left=oecd_bli, right=gdp_per_capita,
                                  left_index=True, right_index=True)
    full_country_stats.sort_values(by="GDP per capita", inplace=True)
    remove_indices = [0, 1, 6, 8, 33, 34, 35]
    keep_indices = list(set(range(36)) - set(remove_indices))
    return full_country_stats[["GDP per capita", 'Life satisfaction']].iloc[keep_indices]

# To plot pretty figures directly within Jupyter
%matplotlib inline
import matplotlib as mpl
mpl.rc('axes', labelsize=14)
mpl.rc('xtick', labelsize=12)
mpl.rc('ytick', labelsize=12)

# Download the data
import urllib.request
DOWNLOAD_ROOT = "https://raw.githubusercontent.com/ageron/handson-ml2/master/"
os.makedirs(datapath, exist_ok=True)
for filename in ("oecd_bli_2015.csv", "gdp_per_capita.csv"):
    print("Downloading", filename)
    url = DOWNLOAD_ROOT + "datasets/lifesat/" + filename
    urllib.request.urlretrieve(url, datapath + filename)

# Code example
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import sklearn.linear_model

# Load the data
oecd_bli = pd.read_csv(datapath + "oecd_bli_2015.csv", thousands=',')
gdp_per_capita = pd.read_csv(datapath + "gdp_per_capita.csv",thousands=',',delimiter='\t',
                             encoding='latin1', na_values="n/a")

# Prepare the data
country_stats = prepare_country_stats(oecd_bli, gdp_per_capita)
X = np.c_[country_stats["GDP per capita"]]
y = np.c_[country_stats["Life satisfaction"]]

# Visualize the data
country_stats.plot(kind='scatter', x="GDP per capita", y='Life satisfaction')
plt.show()

# Select a linear model
model = sklearn.linear_model.LinearRegression()

# Train the model
model.fit(X, y)

# Make a prediction for Cyprus
X_new = [[22587]]  # Cyprus' GDP per capita
print(model.predict(X_new)) # outputs [[ 5.96242338]]

用户提问：It seems to me that the statement oecd_bli = oecd_bli[oecd_bli["INEQUALITY"]=="TOT"] is some sort of "list comprehenson". But I don't understand it. It is apparently a recursive function.

编者回答：

This syntax is a special indexing syntax that works with Pandas DataFrames, NumPy arrays, TensorFlow tensors and a few other libraries.

Here's a simple example:

import numpy as np

a = np.array([10, 20, 30, 40, 50])
i = np.array([False, True, False, True, True]) # is True for every item we want, and otherwise False
print(a[i]) # prints [20 40 50]

Now suppose I only want to keep the even numbers in an array, here's one way to do it:

a = np.array([1, 3, 4, 8, 2, 5, 4])
i = (a % 2 == 0) # this will be equal to array([False, False,  True,  True,  True, False,  True])
print(a[i]) # prints [4 8 2 4]

Now let's look at the line that confused you:

oecd_bli = oecd_bli[oecd_bli["INEQUALITY"]=="TOT"]

First, note that oecd_bli["INEQUALITY"]=="TOT" is a pandas Series equal to True everywhere the "INEQUALITY" feature is equal to "TOT". So oecd_bli[oecd_bli["INEQUALITY"]=="TOT"] is a new pandas DataFrame containing only the rows where the "INEQUALITY" feature is equal to "TOT".

Here's a simplified example:

import pandas as pd

oecd_bli = pd.DataFrame({
    "INEQUALITY": ["a", "b", "TOT", "TOT", "c", "TOT"],
    "Other": [10, 20, 30, 40, 50, 60]
})

print(oecd_bli["INEQUALITY"]=="TOT")
# prints this Pandas Series:
# 0    False
# 1    False
# 2     True 
# 3     True
# 4    False
# 5     True
# Name: INEQUALITY, dtype: bool

oecd_bli = oecd_bli[oecd_bli["INEQUALITY"]=="TOT"]

print(oecd_bli)
# prints:
#  INEQUALITY  Other
# 2        TOT      30
# 3        TOT      40
# 5        TOT      60