Mastering Data Selection in Pandas: A Comprehensive Guide to iloc and loc

Pandas is the most popular and widely used Python library for data manipulation and analysis. A critical part of working with data in pandas is selecting subsets of data from DataFrames, which can be done using the loc and iloc indexers. While they are similar in some ways, it‘s important to understand the differences between the two to use them effectively.

In this guide, we‘ll dive deep into the loc and iloc operators, exploring how to leverage them to their full potential for selecting data in pandas DataFrames. We‘ll cover a range of techniques, from basic selection by labels and positions to more advanced concepts like boolean indexing and slicing. By the end of this article, you‘ll have a complete understanding of when and how to use loc and iloc to manipulate your data with ease.

What‘s the Difference Between loc and iloc?

The fundamental difference between pandas‘ loc and iloc indexers is that loc is label-based, while iloc is integer position based. This means that with loc, you select data based on the labels of the rows and columns, i.e. the index and column names. With iloc, you use the integer position of the rows and columns instead, starting from 0.

In practice, this means that loc takes labels as arguments:

df.loc[row_label, column_label] 

While iloc takes integer positions:

df.iloc[row_position, column_position]

To illustrate, let‘s create a sample DataFrame:

data = {
    ‘name‘: [‘John‘, ‘Alice‘, ‘Bob‘],
    ‘age‘: [25, 31, 45], 
    ‘city‘: [‘New York‘, ‘London‘, ‘Paris‘]
}
df = pd.DataFrame(data, index=[‘a‘, ‘b‘, ‘c‘])

   name  age      city
a  John   25  New York
b  Alice  31    London 
c   Bob   45     Paris

With loc, we reference rows and columns by their labels:

df.loc[‘a‘, ‘city‘]  # Select row ‘a‘, column ‘city‘
# ‘New York‘

With iloc, we use integer positions:

df.iloc[0, 2]  # Select row 0, column 2  
# ‘New York‘

Both return the same value, but in different ways. loc used the row label ‘a‘ and column label ‘city‘, while iloc used the integer positions 0 and 2 for the row and column.

Using loc for Label-based Selection

The loc indexer allows you to select data from a DataFrame by labels. You can select single rows or columns, multiple rows or columns, or a combination of both.

Selecting Rows by Label

To select a single row by its label, you can pass the row label to loc:

df.loc[‘a‘]

name        John
age           25
city    New York
Name: a, dtype: object

You can also select multiple rows by passing a list of labels:

df.loc[[‘a‘, ‘c‘]]

   name  age      city
a  John   25  New York
c   Bob   45     Paris

Selecting Columns by Label

Similarly, you can select a single column or multiple columns by passing the column labels:

df.loc[:, ‘age‘] 

a    25
b    31
c    45
Name: age, dtype: int64

df.loc[:, [‘name‘, ‘city‘]]

   name      city
a  John  New York
b  Alice   London
c   Bob     Paris

Here, the : before the comma means to select all rows.

Selecting Rows and Columns Together

You can combine row and column selection by passing both row and column labels to loc:

df.loc[[‘a‘, ‘c‘], [‘name‘, ‘city‘]]

   name      city
a  John  New York 
c   Bob     Paris

Boolean Indexing with loc

Another powerful feature of loc is boolean indexing, where you can pass a boolean Series or array to select rows that meet a certain condition.

For example, to select all rows where the age is greater than 30:

df.loc[df[‘age‘] > 30]

   name  age   city
b  Alice  31  London
c   Bob   45   Paris

You can combine multiple conditions using the & (and) and | (or) operators:

df.loc[(df[‘age‘] > 30) & (df[‘city‘] == ‘Paris‘)]  

  name  age   city
c  Bob   45  Paris

Setting Values with loc

In addition to selecting data, you can also use loc to set values in a DataFrame.

To set a single value:

df.loc[‘a‘, ‘age‘] = 26

To set multiple values:

df.loc[[‘a‘, ‘c‘], ‘city‘] = ‘Tokyo‘

This will update the ‘city‘ for rows ‘a‘ and ‘c‘ to ‘Tokyo‘.

Using iloc for Integer Position based Selection

The iloc indexer allows you select data from a DataFrame by integer position. Like loc, you can select single rows or columns, multiple rows or columns, or a combination of both.

Selecting Rows by Integer Position

To select a single row by its integer position, you can pass the integer to iloc:

df.iloc[0]  

name        John
age           25 
city    New York
Name: a, dtype: object

To select multiple rows, pass a list of integers:

df.iloc[[0, 2]]

   name  age      city
a  John   25  New York
c   Bob   45     Tokyo  

Selecting Columns by Integer Position

You can select columns by passing the integer positions of the columns:

df.iloc[:, 1]

a    25
b    31
c    45 
Name: age, dtype: int64

df.iloc[:, [0, 2]] 

   name      city
a  John  New York
b  Alice   London
c   Bob     Tokyo

Selecting Rows and Columns Together

You can select specific rows and columns by passing lists of integers for both:

df.iloc[[0, 2], [0, 2]]

   name      city 
a  John  New York
c   Bob     Tokyo

Boolean Indexing with iloc

While iloc is primarily used for integer position based selection, you can still use boolean indexing with it by passing a boolean array or Series:

df.iloc[(df[‘age‘] > 30).values]

   name  age   city
b  Alice  31  London  
c   Bob   45   Tokyo

Note that we used .values to convert the boolean Series to a NumPy array, as iloc does not accept boolean Series.

Setting Values with iloc

Like with loc, you can set values using iloc by specifying the integer positions.

To set a single value:

df.iloc[0, 1] = 27

To set multiple values:

df.iloc[[0, 2], 2] = ‘Berlin‘ 

This will update the values in the third column (index 2) for the first and third rows (indices 0 and 2).

Using Negative Integers with iloc

One useful feature of iloc is that it accepts negative integers for indexing. Negative integers count backwards from the end of the axis.

For example, to select the last row:

df.iloc[-1]

name     Bob
age       45
city    Berlin
Name: c, dtype: object 

You can mix positive and negative integers:

df.iloc[[-1, 0]]

   name  age     city
c   Bob   45   Berlin
a  John   27  Berlin

Using Lists or Arrays as Indices

With iloc, you can use lists or NumPy arrays of integers to select data.

rows = np.array([0, 1])
cols = np.array([1, 2]) 
df.iloc[rows, cols]

   age     city
a   27  Berlin  
b   31   London

This can be useful when you have the integer positions stored in variables or coming from some other part of your code.

Slicing with iloc

One of the most powerful features of iloc is slicing. You can pass slice objects to iloc to select ranges of rows or columns.

To select a range of rows:

df.iloc[0:2]  # Selects rows 0 and 1

   name  age     city
a  John   27   Berlin
b  Alice  31   London

To select a range of columns:

df.iloc[:, 1:3]  # Selects columns 1 and 2

   age     city
a   27  Berlin
b   31   London
c   45  Berlin

You can combine row and column slicing:

df.iloc[1:3, 0:2]  # Selects rows 1 and 2, columns 0 and 1

   name  age
b  Alice  31
c   Bob   45

Slicing with iloc is inclusive of the start bound and exclusive of the stop bound, just like regular Python slicing.

Getting Views vs Copies with iloc

One important thing to note about iloc is that it can return either a view or a copy of the data, depending on the types of indices used.

• If you use only integers, you will get a copy.
• If you use only slices, you will get a view.
• If you use a mixture of integers and slices, you will get a copy.

This is important to keep in mind if you plan to modify the returned data. If it‘s a view, any modifications will affect the original DataFrame. If it‘s a copy, modifications will not affect the original DataFrame.

You can use the is_copy attribute to check if the returned data is a view or a copy:

df.iloc[0:2].is_copy  # Returns a view
# None

df.iloc[[0, 2]].is_copy  # Returns a copy  
# True

Summary of Differences Between loc and iloc

Here‘s a summary table comparing the key differences between loc and iloc:

Feature	loc	iloc
Indexing	Label-based	Integer position based
Input types	Labels (strings, booleans, etc.)	Integers
Negative indexing	Not allowed	Allowed
Slicing	Inclusive of both start and stop bounds	Inclusive of start bound, exclusive of stop bound
Boolean indexing	Allowed	Allowed, but requires .values
Setting values	Allowed	Allowed
Returns views or copies	Always returns views	Can return views or copies

Conclusion

In this comprehensive guide, we‘ve explored the loc and iloc indexers in pandas, the two primary ways to select data from DataFrames. We‘ve seen how loc is used for label-based selection and iloc for integer position based selection, and we‘ve delved into a variety of techniques for each, including boolean indexing, slicing, and setting values.

Understanding the differences and use cases for loc and iloc is crucial for effective data manipulation in pandas. With the knowledge you‘ve gained from this article, you‘ll be able to confidently select and manipulate data in your DataFrames using these powerful tools.

Remember, the choice between loc and iloc depends on whether you want to select data based on labels or integer positions. If you‘re working with labeled data and want to maintain those labels in your selections, use loc. If you want to select data based on its integer position, regardless of the labels, use iloc.

Happy data wrangling!