Converting Object Data Types to Numeric and Integer in Pandas

When working with datasets in pandas, it’s common to encounter columns with object data types that actually contain numeric or integer values. This can happen when reading data from various sources, such as SQL queries or CSV files. In this tutorial, we’ll explore the different methods for converting object data types to numeric and integer in pandas.

Introduction to Pandas Data Types

Pandas is a powerful library for data manipulation and analysis in Python. It provides data structures like Series (1-dimensional labeled array) and DataFrame (2-dimensional labeled data structure with columns of potentially different types). When reading data into a pandas DataFrame, the library automatically assigns a data type to each column based on its content.

Converting Object Data Type to Integer

One common scenario is when you have a column that contains integer values but is assigned an object data type. To convert such a column to an integer data type, you can use the astype() method. However, if the column contains any non-numeric values (including NaN), this will raise an error.

import pandas as pd

# Example DataFrame with object data type
df = pd.DataFrame({
    'purchase': ['1', '2', '3']
})

print(df['purchase'].dtype)  # Output: object

# Convert to integer using astype()
df['purchase'] = df['purchase'].astype(int)

print(df['purchase'].dtype)  # Output: int64

Handling Non-Numeric Values with pd.to_numeric()

If your column contains non-numeric values, you can use the pd.to_numeric() function to convert it to a numeric data type. This function provides an errors parameter that allows you to specify how to handle errors. For example, you can set errors='coerce' to convert non-numeric values to NaN.

import pandas as pd

# Example DataFrame with object data type and non-numeric value
df = pd.DataFrame({
    'purchase': ['1', '2', 'abc']
})

print(df['purchase'].dtype)  # Output: object

# Convert to numeric using pd.to_numeric()
df['purchase'] = pd.to_numeric(df['purchase'], errors='coerce')

print(df['purchase'].dtype)  # Output: float64

Using convert_dtypes() for Nullable Integer Types

In pandas version 1.0 and later, you can use the convert_dtypes() method to convert columns to their corresponding nullable integer types. This is useful when working with datasets that contain missing values.

import pandas as pd

# Example DataFrame with object data type and NaN value
df = pd.DataFrame({
    'purchase': [1, 2, None]
}, dtype=object)

print(df['purchase'].dtype)  # Output: object

# Convert to nullable integer using convert_dtypes()
df['purchase'] = df['purchase'].convert_dtypes()

print(df['purchase'].dtype)  # Output: Int64

Conclusion

Converting object data types to numeric and integer in pandas is a common task when working with datasets. By using the astype(), pd.to_numeric(), and convert_dtypes() methods, you can efficiently convert your columns to the desired data type while handling non-numeric values and missing data.