Right, and if this is happening inside a loop or in a batch process, frequent use of concat
can be a performance bottleneck. Instead, it’s better to collect all your rows in a list and then use concat()
once at the end. Here’s an example:
rows = [existing_row1, existing_row2, new_row]
df = pd.DataFrame(rows)
Or even store the new rows in a list and convert them to a DataFrame later:
new_data = pd.DataFrame(new_rows_list)
df = pd.concat([df, new_data], ignore_index=True)
This method avoids the costly reallocation that comes with repeated appending. It’s a big performance win, especially with larger datasets.