Libraries are an essential component for developers to program in a simple way, avoiding code duplication and minimizing errors.
They also enable greater agility by reducing development time and costs.
NumPy is a fundamental package for doing scientific computing in Python. It is a library that provides a multidimensional array (equivalent to a list, but more optimal), several derived objects and a variety of routines for fast operations on arrays, including mathematical operations, logic, dimension manipulation, sorting, selection, I/O, discrete Fourier transforms, basic linear algebra, basic statistical operations, random simulation and much more.
import numpy as np
np.array([1, 2, 3, 4, 5, 6])
array([1, 2, 3, 4, 5, 6])
np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]])
array([[ 1, 2, 3, 4], [ 5, 6, 7, 8], [ 9, 10, 11, 12]])
I can also create an array through a structure that iterates the values within a range:
a = np.arange(6)
print(a)
[0 1 2 3 4 5]
b = a.reshape(3, 2)
print(b)
[[0 1] [2 3] [4 5]]
Slicing works just like a list
b[0:2]
array([[0, 1], [2, 3]])
b[0:2, 1]
array([1, 3])
c = np.array([[1 , 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]])
print(c[c < 5])
[1 2 3 4]
print(c[(c > 2) & (c < 11)])
[ 3 4 5 6 7 8 9 10]
Generate a random sample
np.random.choice(5, 3)
array([0, 1, 0])
np.random.choice(a, 3)
array([0, 2, 2])
np.random.rand(3,2)
array([[0.55673422, 0.3258215 ], [0.50082954, 0.20422156], [0.40866089, 0.29196281]])
To see all the randomness that you can get, go here
print(f'{c.max()=}, {c.min()=}, {c.mean()=}, {c.sum()=}')
c.max()=12, c.min()=1, c.mean()=6.5, c.sum()=78
print(f'{c.min(axis=0)=}, {c.min(axis=1)=}')
c.min(axis=0)=array([1, 2, 3, 4]), c.min(axis=1)=array([1, 5, 9])
reversed_arr = np.flip(c)
print(reversed_arr)
[[12 11 10 9] [ 8 7 6 5] [ 4 3 2 1]]
concat_arrays = np.concatenate((c, reversed_arr))
print(concat_arrays)
[[ 1 2 3 4] [ 5 6 7 8] [ 9 10 11 12] [12 11 10 9] [ 8 7 6 5] [ 4 3 2 1]]
concat_on_two_d = np.c_[np.array([1,2,3]), np.array([4,5,6])]
print(concat_on_two_d)
[[1 4] [2 5] [3 6]]
Pandas is a Python library used to work with data sets. It has functions to analyze, clean, explore and manipulate data.
We are going to see some functions, you can always look for more information in the official pandas documentation (here) or you can look for a cheat sheet to have some things at hand (for example this).
In general (but not exclusively) you work with two types of data structures to manipulate the data:
import pandas as pd
A series can be created from a list
s = pd.Series([1, 3, 5, np.nan, 6, 8])
s
0 1.0 1 3.0 2 5.0 3 NaN 4 6.0 5 8.0 dtype: float64
It can also be created from other functions of the same library: Pandas handles dates very well, it has many functions to work with them. An example of this is that we can generate a date range easily.
fechas = pd.date_range("20221123", periods=6)
fechas
DatetimeIndex(['2022-11-23', '2022-11-24', '2022-11-25', '2022-11-26', '2022-11-27', '2022-11-28'], dtype='datetime64[ns]', freq='D')
A dataframe can be generated by passing values, indexes and columns to it.
df = pd.DataFrame(data=np.random.randn(6, 4), index=fechas, columns=list("ABCD"))
df
A | B | C | D | |
---|---|---|---|---|
2022-11-23 | -0.970778 | -1.379294 | -0.847248 | -1.245905 |
2022-11-24 | 2.021943 | 1.178061 | -2.219878 | -0.848414 |
2022-11-25 | 1.337639 | -0.258540 | -1.428744 | 0.482594 |
2022-11-26 | 0.271419 | 1.199697 | 1.600485 | -0.402905 |
2022-11-27 | -0.000121 | 0.461117 | -1.497202 | 1.395536 |
2022-11-28 | -0.054047 | 0.228183 | -0.092019 | 1.295134 |
Another way to create it can be through a dictionary.
df2 = pd.DataFrame(
{
"A": 1.0,
"B": fechas[:4],
"C": pd.Series(1, index=list(range(4)), dtype="float32"),
"D": np.array([3] * 4, dtype="int32"),
"E": pd.Categorical(["test", "train", "test", "train"]),
"F": "foo",
}
)
df2
A | B | C | D | E | F | |
---|---|---|---|---|---|---|
0 | 1.0 | 2022-11-23 | 1.0 | 3 | test | foo |
1 | 1.0 | 2022-11-24 | 1.0 | 3 | train | foo |
2 | 1.0 | 2022-11-25 | 1.0 | 3 | test | foo |
3 | 1.0 | 2022-11-26 | 1.0 | 3 | train | foo |
Other options: pd.from_records
or pd.from_dict
.
df['A'] # equivalent a df.A
2022-11-23 -0.970778 2022-11-24 2.021943 2022-11-25 1.337639 2022-11-26 0.271419 2022-11-27 -0.000121 2022-11-28 -0.054047 Freq: D, Name: A, dtype: float64
I select some indexes and some columns according to their names through loc[idx, col]
df.loc["20221123":"20221125", ["A", "C"]]
A | C | |
---|---|---|
2022-11-23 | -0.970778 | -0.847248 |
2022-11-24 | 2.021943 | -2.219878 |
2022-11-25 | 1.337639 | -1.428744 |
I can also use loc
to get a single value, specifying only one index and one column
I select some indexes and some columns according to their indexes through iloc[idx, col]
df.iloc[[1, 2, 4], [0, 2]]
A | C | |
---|---|---|
2022-11-24 | 2.021943 | -2.219878 |
2022-11-25 | 1.337639 | -1.428744 |
2022-11-27 | -0.000121 | -1.497202 |
df[df["A"] > 0]
A | B | C | D | |
---|---|---|---|---|
2022-11-24 | 2.021943 | 1.178061 | -2.219878 | -0.848414 |
2022-11-25 | 1.337639 | -0.258540 | -1.428744 | 0.482594 |
2022-11-26 | 0.271419 | 1.199697 | 1.600485 | -0.402905 |
df[(df["A"] > 0) & (df['B'] < 0)]
A | B | C | D | |
---|---|---|---|---|
2022-11-25 | 1.337639 | -0.25854 | -1.428744 | 0.482594 |
df
A | B | C | D | |
---|---|---|---|---|
2022-11-23 | -0.970778 | -1.379294 | -0.847248 | -1.245905 |
2022-11-24 | 2.021943 | 1.178061 | -2.219878 | -0.848414 |
2022-11-25 | 1.337639 | -0.258540 | -1.428744 | 0.482594 |
2022-11-26 | 0.271419 | 1.199697 | 1.600485 | -0.402905 |
2022-11-27 | -0.000121 | 0.461117 | -1.497202 | 1.395536 |
2022-11-28 | -0.054047 | 0.228183 | -0.092019 | 1.295134 |
df2
A | B | C | D | E | F | |
---|---|---|---|---|---|---|
0 | 1.0 | 2022-11-23 | 1.0 | 3 | test | foo |
1 | 1.0 | 2022-11-24 | 1.0 | 3 | train | foo |
2 | 1.0 | 2022-11-25 | 1.0 | 3 | test | foo |
3 | 1.0 | 2022-11-26 | 1.0 | 3 | train | foo |
df_joined = df.join(df2.set_index('B'), how='left', lsuffix='_left', rsuffix='_right')
df_joined
A_left | B | C_left | D_left | A_right | C_right | D_right | E | F | |
---|---|---|---|---|---|---|---|---|---|
2022-11-23 | -0.970778 | -1.379294 | -0.847248 | -1.245905 | 1.0 | 1.0 | 3.0 | test | foo |
2022-11-24 | 2.021943 | 1.178061 | -2.219878 | -0.848414 | 1.0 | 1.0 | 3.0 | train | foo |
2022-11-25 | 1.337639 | -0.258540 | -1.428744 | 0.482594 | 1.0 | 1.0 | 3.0 | test | foo |
2022-11-26 | 0.271419 | 1.199697 | 1.600485 | -0.402905 | 1.0 | 1.0 | 3.0 | train | foo |
2022-11-27 | -0.000121 | 0.461117 | -1.497202 | 1.395536 | NaN | NaN | NaN | NaN | NaN |
2022-11-28 | -0.054047 | 0.228183 | -0.092019 | 1.295134 | NaN | NaN | NaN | NaN | NaN |
Try alsomerge
group_df = pd.DataFrame(
{
"A": ["foo", "bar", "foo", "bar", "foo", "bar", "foo", "foo"],
"B": ["one", "one", "two", "three", "two", "two", "one", "three"],
"C": np.random.randn(8),
"D": np.random.randn(8),
}
)
group_df
A | B | C | D | |
---|---|---|---|---|
0 | foo | one | -0.713073 | 1.143117 |
1 | bar | one | 0.691881 | -0.443244 |
2 | foo | two | 0.253103 | 1.856946 |
3 | bar | three | -0.537566 | -0.159498 |
4 | foo | two | 0.676906 | 0.751306 |
5 | bar | two | -0.057244 | -1.167156 |
6 | foo | one | -1.266099 | 1.079533 |
7 | foo | three | -1.025097 | -0.740996 |
group_df.groupby('A').size()
A bar 3 foo 5 dtype: int64
df_joined.groupby(['E', 'F']).sum()
A_left | B | C_left | D_left | A_right | C_right | D_right | ||
---|---|---|---|---|---|---|---|---|
E | F | |||||||
test | foo | 0.366861 | -1.637834 | -2.275991 | -0.763312 | 2.0 | 2.0 | 6.0 |
train | foo | 2.293362 | 2.377758 | -0.619393 | -1.251319 | 2.0 | 2.0 | 6.0 |
data = pd.read_csv('iris.csv', index_col=0)
data
sepal length (cm) | sepal width (cm) | petal length (cm) | petal width (cm) | species | |
---|---|---|---|---|---|
0 | 5.1 | 3.5 | 1.4 | 0.2 | 0.0 |
1 | 4.9 | 3.0 | 1.4 | 0.2 | 0.0 |
2 | 4.7 | 3.2 | 1.3 | 0.2 | 0.0 |
3 | 4.6 | 3.1 | 1.5 | 0.2 | 0.0 |
4 | 5.0 | 3.6 | 1.4 | 0.2 | 0.0 |
... | ... | ... | ... | ... | ... |
145 | 6.7 | 3.0 | 5.2 | 2.3 | 2.0 |
146 | 6.3 | 2.5 | 5.0 | 1.9 | 2.0 |
147 | 6.5 | 3.0 | 5.2 | 2.0 | 2.0 |
148 | 6.2 | 3.4 | 5.4 | 2.3 | 2.0 |
149 | 5.9 | 3.0 | 5.1 | 1.8 | 2.0 |
150 rows × 5 columns
df.to_csv('prueba.csv')
This is a library that allows us to make simple graphs in a very simple way. At the same time, it also allows us, with a little more effort, to make very complex graphs.
It is based on NumPy arrays and allows us to make line charts, bar charts, scatter charts, histograms and much more.
Pyplot is a Matplotlib module that provides an interface similar to MATLAB. Pyplot provides functions that interact with the figure, i.e. it creates a figure, decorates the plot with labels and creates a plot area on a figure.
import matplotlib.pyplot as plt
x = [1, 2, 3, 4]
y = [1, 4, 9, 16]
plt.plot(x, y)
plt.title('Mi primer gráfico')
Text(0.5, 1.0, 'Mi primer gráfico')
plt.plot(x, y, color='blue', linewidth=3, linestyle='--')
plt.scatter(x,y, color='red')
<matplotlib.collections.PathCollection at 0x7fca1bb40df0>
Sometimes we will need to do things a bit more complex (several graphics in one, two graphics side by side, and many other things), for that we will need to learn what figure and axis are.
fig, ax = plt.subplots(figsize=(5, 2.7))
x = np.linspace(0, 2, 100)
ax.plot(x, x, label='lineal')
ax.plot(x, x**2, label='cuadrática')
ax.plot(x, x**3, label='cubica')
ax.set_xlabel('eje x')
ax.set_ylabel('eje y')
ax.set_title('Un título')
ax.legend() # necesario para que se muestren los lables de la data que estoy graficando
<matplotlib.legend.Legend at 0x7fca1ddbfaf0>
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(8,2))
ax1.plot([1, 2, 3, 4], [1, 2, 3, 4])
ax1.plot([1, 2, 3, 4], [4, 3, 2, 1])
ax2.plot([1, 2, 3, 4], [1, 1, 1, 1])
[<matplotlib.lines.Line2D at 0x7fca1baae710>]
Try to add sharex='all'
or sharey='all'
on subplots arguments
Load the file called 'iris.csv'
Display the pandas.Dataframe structure.
Select two columns and make a scatter plot of both (one as x, the other as y).
Choose a column, count the number of times a value is repeated in that column and make a bar chart. To do this
.size()
function (after groupby). This will result in a .pd.Series
.bar
function inside pyplot to make the bar chart. Assign the values of series.index
and series.values
to the x
and height
variables respectively.Use the same values from the previous point to make a histogram.
.plot.hist()
.