In this article we will discuss an efficient solution to read the contents of a text or CSV file in reverse order i.e. either line by line or get them as a reversed list.
Read a file line by line in reversed order using python
An efficient solution to read a file in reverse order is,
Start reading the file from last and continue till the start of the file i.e. in reverse order. As soon as it encounter any ‘n’ then it means, a complete line is read. Then yield that line and continue reading in the reverse direction until the top of the file is reached.
We have implemented this logic to a function,
Frequently Asked:
import os
def read_reverse_order(file_name):
# Open file for reading in binary mode
with open(file_name, 'rb') as read_obj:
# Move the cursor to the end of the file
read_obj.seek(0, os.SEEK_END)
# Get the current position of pointer i.e eof
pointer_location = read_obj.tell()
# Create a buffer to keep the last read line
buffer = bytearray()
# Loop till pointer reaches the top of the file
while pointer_location >= 0:
# Move the file pointer to the location pointed by pointer_location
read_obj.seek(pointer_location)
# Shift pointer location by -1
pointer_location = pointer_location -1
# read that byte / character
new_byte = read_obj.read(1)
# If the read byte is new line character then it means one line is read
if new_byte == b'n':
# Fetch the line from buffer and yield it
yield buffer.decode()[::-1]
# Reinitialize the byte array to save next line
buffer = bytearray()
else:
# If last read character is not eol then add it in buffer
buffer.extend(new_byte)
# As file is read completely, if there is still data in buffer, then its the first line.
if len(buffer) > 0:
# Yield the first line too
yield buffer.decode()[::-1]
This function accepts the name of the file as an argument and then yields the lines of file from bottom to top.
How does this function work?
It opens the file in binary read mode and moves the cursor to the end of file using file.seek(). Then it starts reading each byte from the end of the file until the start of the file i.e. in reverse direction and save those bytes in a buffer. While reading each byte, as soon as it encounters a new line character ‘n’, it means a line is read successfully. Then it reverses the buffer and yields the content of buffer i.e. a complete line. After that it re initializes the buffer and continue reading next bytes from the file in reverse direction till the top of the file is reached and keeps on yielding lines till top of file.
Let’s use this function,
Best Resources to Learn Python:
Suppose we have a file ‘sample.txt’ and its contents are,
Hello this is a sample file It contains sample text Dummy Line A Dummy Line B Dummy Line C This is the end of file
Read the contents of a ‘sample.txt’ in reverse order line by line,
# Iterate over the file in reverse order using for loop
for line in read_reverse_order('sample.txt'):
print(line)
Output:
This is the end of file Dummy Line C Dummy Line B Dummy Line A It contains sample text Hello this is a sample file
As we are reading one byte at a time and keeping only last line in buffer, therefore even if the file is large, our solution will be efficient.
Get a list of all lines of file in reversed order using python
Instead of reading line by line, suppose we want to get all the lines of a file as a list but in reverse order.
To do that we have created a function that reads all lines of a file in list and then returns a reversed list,
def read_reverse_order_2(file_name):
"""Read a file in reverse order line by line"""
# Open file in read mode
with open(file_name, 'r') as read_obj:
# get all lines in a file as list
lines = read_obj.readlines()
lines = [line.strip() for line in lines]
# reverse the list
lines = reversed(lines)
# Return the list with all lines of file in reverse order
return lines
Let’s use this function to get a reversed list of all lines of a file ‘sample.txt’ and then iterate over that list,
# Get a list of lines in file as reverse order
lines_in_reverse_order = read_reverse_order_2('sample.txt')
# iterate over the lines in list
for line in lines_in_reverse_order:
print(line)
Output:
This is the end of file Dummy Line C Dummy Line B Dummy Line A It contains sample text Hello this is a sample file
Here we got all the lines of a file as a list in reverse order, then we iterated over that list.
If you just want to iterate over the lines of a file in reverse order then this is not an efficient solution, because it keeps all the lines in memory and if file is large like in GBs then it will create issues. So, use this solution with small files only and for large files prefer the first solution mentioned above. It was a little complex but was surely an efficient solution.
The complete example is as follows,
import os
def read_reverse_order_2(file_name):
"""Read a file in reverse order line by line"""
# Open file in read mode
with open(file_name, 'r') as read_obj:
# get all lines in a file as list
lines = read_obj.readlines()
lines = [line.strip() for line in lines]
# reverse the list
lines = reversed(lines)
# Return the list with all lines of file in reverse order
return lines
def read_reverse_order(file_name):
# Open file for reading in binary mode
with open(file_name, 'rb') as read_obj:
# Move the cursor to the end of the file
read_obj.seek(0, os.SEEK_END)
# Get the current position of pointer i.e eof
pointer_location = read_obj.tell()
# Create a buffer to keep the last read line
buffer = bytearray()
# Loop till pointer reaches the top of the file
while pointer_location >= 0:
# Move the file pointer to the location pointed by pointer_location
read_obj.seek(pointer_location)
# Shift pointer location by -1
pointer_location = pointer_location -1
# read that byte / character
new_byte = read_obj.read(1)
# If the read byte is new line character then it means one line is read
if new_byte == b'n':
# Fetch the line from buffer and yield it
yield buffer.decode()[::-1]
# Reinitialize the byte array to save next line
buffer = bytearray()
else:
# If last read character is not eol then add it in buffer
buffer.extend(new_byte)
# As file is read completely, if there is still data in buffer, then its the first line.
if len(buffer) > 0:
# Yield the first line too
yield buffer.decode()[::-1]
def main():
print('**** Read a file in reversed order line by line ****')
# Iterate over the file in reverse order using for loop
for line in read_reverse_order('sample.txt'):
print(line)
print('**** Get a list of all lines in file in reversed order****')
# Get a list of lines in file as reverse order
lines_in_reverse_order = read_reverse_order_2('sample.txt')
# iterate over the lines in list
for line in lines_in_reverse_order:
print(line)
if __name__ == '__main__':
main()
Output
**** Read a file in reversed order line by line **** This is the end of file Dummy Line C Dummy Line B Dummy Line A It contains sample text Hello this is a sample file **** Get a list of all lines in file in reversed order**** This is the end of file Dummy Line C Dummy Line B Dummy Line A It contains sample text Hello this is a sample file