How to Split a big CSV File into Smaller files based on a word using python

Hello here, I was tasked to break down a large file into smaller files whenever i reach on the “Total” row, I searched for a few solutions and came up with this python script. Please leave your comment on this.

 

import csv

def split_csv_by_keyword(filename, keyword):
  with open(filename, "r") as file:
    reader = csv.reader(file)
    data = []
    for row in reader:
      data.append(row)
    file_count = 1
    current_file_rows = []
    for row in data:
      if keyword in row[0]:
        with open(f"{filename.split('.')[0]}_{file_count}.csv", "w") as outfile:
          writer = csv.writer(outfile)
          writer.writerows(current_file_rows)
        file_count += 1
        current_file_rows = []
      else:
        current_file_rows.append(row)
    if current_file_rows:
      with open(f"{filename.split('.')[0]}_{file_count}.csv", "w") as outfile:
        writer = csv.writer(outfile)
        writer.writerows(current_file_rows)

# Example usage:
split_csv_by_keyword("may_2022.csv", "Total")

For those interested in Raspberry Pi, Hit the link and take a crush course that can take you from beginner to pro.

Leave a Reply

Your email address will not be published. Required fields are marked *