Copy Files and Directories in Python

In this Python tutorial, you’ll learn how to use various functions available in the os, shutil, and subprocess modules to copy files and folders from one location to another.

After reading this article, you’ll learn: –

• How to copy files in Python using shutil module’s copy(), copy2(), copyfiles(), copyfileobj() methods
• The OS and subprocess module to copy files using the underlying operating system’s shell command (copy in Windows or cp in UNIX.)
• How to copy all files from a directory
• Copy an entire directory recursively

Steps to Copy a File in Python

Example: Copy Single File

In this example, we are copying the profit.txt file from the report folder to the account folder.

import shutil

src_path = r"E:\demos\files\report\profit.txt"
dst_path = r"E:\demos\files\account\profit.txt"
shutil.copy(src_path, dst_path)
print('Copied')

Copy All Files From A Directory

Sometimes we want to copy all files from one directory to another. Follow the below steps to copy all files from a directory.

• Store the source and destination directory path into two variables
• Get the list of all files present in the source folder using the os.listdir() function. It returns a list containing the names of the files and folders in the given directory.
• Iterate over the list using a for loop to get the individual filenames
• In each iteration, concatenate the current file name with the source folder path
• Now use the shutil.copy() method to copy the current file to the destination folder path.

Example:

import os
import shutil

source_folder = r"E:\demos\files\reports\\"
destination_folder = r"E:\demos\files\account\\"

# fetch all files
for file_name in os.listdir(source_folder):
    # construct full file path
    source = source_folder + file_name
    destination = destination_folder + file_name
    # copy only files
    if os.path.isfile(source):
        shutil.copy(source, destination)
        print('copied', file_name)

Output:

copied expense.txt
copied profit.txt
copied revenue.txt

Copy Entire Directory

Sometimes we need to copy an entire folder, including all files and subfolder contained in it. Use the copytree() method of a shutil module to copy the directory recursively.

shutil.copytree(src, dst, symlinks=False, ignore=None, copy_function=copy2, ignore_dangling_symlinks=False, dirs_exist_ok=False

• This method recursively copy an entire directory tree rooted at src to a directory named dst and return the destination directory.
• The dirs_exist_ok dictates whether to raise an exception in case dst or any missing parent directory already exists.
• Use the copy_function parameter to pass any of the four functions the shutil module provides to copy files.

Example: Let’s to how to copy the report directory and all its content into an account directory.

import shutil

source_dir = r"E:\demos\files\reports"
destination_dir = r"E:\demos\files\account"
shutil.copytree(source_dir, destination_dir)

Shutil Module to Copy files

Python shutil module offers many high-end functions to perform copy and removal of files.

These functions provide an optimized way to copy files and thereby save time in performing the unnecessary task of opening, reading, writing, and closing the files when there is no processing required in that file.

The shutil module offers the following four functions to copy files.

Function	Description
shutil.copyfileobj()	Copy the file object from a source to the destination. (Need to open both source and destination path)
shutil.copyfile()	Copy the contents of one file to another file. The metadata of the file will not be copied.
shutil.copy()	copy the contents of the source file to the destination file along with metadata.
shutil.copy2()	copy the additional metadata namely timestamps of the source file to the destination

To decide which function to use for copying a file, we have to consider the following points.

• For example, If your application uses the shared resources, you should copy a file in blocking mode instead of asynchronously.
• Platform portability. If your application runs on a different operating system, you must write a code that will run on a different OS without any issue.
• The function is resource-intensive as some copy functions involve opening the file, reading the contents, writing and closing the file. so choose as per your need
• Do you want to copy metadata along with file content?

The shutil.copyfile() method

The copyfile() method is used to copy the contents of one file to another file. The metadata of the file will not be copied.

shutil.copyfile(src, dst, *, follow_symlinks=True)

• fsrc : The source file location (path) whose contents need to be copied.
• fdst : The destination file location (path) where the contents from the source file will be pasted.
• follow_symlinks : The default value for this is true. If it is false and the source file location is a symbolic link then a new symbolic link will be created instead of copying from the file.

Note: The metadata of the file will not be copied while using this method.

Example:

import shutil

src_folder = r"E:\demos\files\reports"
dst_folder = r"E:\demos\files\account"

# file names
src_file = src_folder + "\profit.txt"
dst_file = dst_folder + "\profit.txt"

shutil.copyfile(src_file, dst_file)
print('Copied')

Our code copied the ‘profit.txt’ file in the destination directory. Here is a list of the files present in the destination directory:

• profit.txt
• revenue.txt

Use the os.listdir(dst_folder) function to list all files present in the destination directory to verify the result.

The following are the important points

• Both the source and destination should be a file. If the destination is a directory then IsADirectoryError will be raised.
• If the source and destination point to the same file then the SameFileError will be raised.
• If the destination exists but with a different name compared to the source file then the contents of the destination file will be overwritten.

To avoid the above errors always wrap your code in the try-except block. Refer the following example.

import os
import shutil
from shutil import SameFileError

src_folder = r"E:\demos\files\reports"
dst_folder = r"E:\demos\files\account"

# printing the contents of the destination folder
print("Destination folder before copying::", os.listdir(dst_folder))

# file names
src_file = src_folder + "\profit.txt"
dst_file = dst_folder + "\profit.txt"

try:
    # copy file
    shutil.copyfile(src_file, dst_file)
    # destination folder after copying
    print("Destination after copying", os.listdir(dst_folder))
except SameFileError:
    print("We are trying to copy the same File")
except IsADirectoryError:
    print("The destination is a directory")

Output

Destination folder before copying:: ['expense.txt', 'revenue.txt']
Destination after copying ['expense.txt', 'profit.txt', 'revenue.txt']

The shutil.copy() method

The shutil module has one more method to copy the file called copy() method which will copy the contents of the source file to the destination file.

shutil.copy(src, dst, *, follow_symlinks=True)

Similar to the copyfile() method this method will throw the ‘SameFileError‘ if the source and destination point to the same file.

Consider the following example where we try to copy the file ‘profit.txt’ from the source src to the destination dst directory.

import shutil

src_path = r"E:\demos\files\reports\profit.txt"
dst_path = r"E:\demos\files_demos\account\profit.txt"
shutil.copy(src_path, dst_path)
print('Copied')

Note: If the source file is not found in the location then the copy() method can’t be executed and an IOError will be raised. If the destination folder is not writable then ‘PermissionError‘ will be raised.

copy() vs copyfiles()

While the copy() and copyfile() methods in the shutil module work the same way in copying files from one location to another there are some significant differences between them.

If the destination path is ‘file’ or ‘directory’ then the copy() method will create a file in the name of the source file.

• The copy() method saves the additional metadata information about the permission modes from the source to the destination file. The copyfile() doesn’t save any metadata information.
• As the copy() method makes one additional call to save the permission modes; it is slower when compared to the copyfile() method.

The copy() method calls the copyfile() and copymode() inside its implementation internally. But the copyfile() only calls the copyfileobj() inside its implementation

The shutil.copy2() method

In shutil there is one more method called copy2() which could be used to copy the file from one location to another.

shutil.copy2(src, dst, *, follow_symlinks=True)

While the copy() and the copy2() methods both copy the contents of the file along with the metadata there are two significant differences between these two methods.

• The copy2() method can be used to copy the additional metadata namely timestamps of the source to the destination. This is in addition to the permission modes which the copy() method saves.
• The copy2() method calls the copystat() method to get the timestamp information while the copy() method saves the permission modes from the copymode()

The scenarios that this method throws the ‘SameFileError’ are similar to the copy() method.We can see how to use this method to copy the file named ‘Sample.txt’ from the source to the destination.

import os
import shutil

src_path = r"E:\demos\files\reports\profit.txt"
dst_path = r"E:\demos\files\account\profit.txt"

# metadata of source file
metadata_src = os.stat(src_path)
print("Metadata source:", metadata_src)

# copy file
shutil.copy2(src_path, dst_path)
print('Copied')
metadata_dst = os.stat(dst_path)
print("Metadata destination:", metadata_dst)

Ouput

Metadata source: os.stat_result(st_mode=33206, st_ino=4222124650735236, st_dev=3202242018, st_nlink=1, st_uid=0, st_gid=0, st_size=0, st_atime=1624768574, st_mtime=1623330684, st_ctime=1624768574)

Copied

Metadata destination: os.stat_result(st_mode=33206, st_ino=5348024557577894, st_dev=3202242018, st_nlink=1, st_uid=0, st_gid=0, st_size=0, st_atime=1624768574, st_mtime=1623330684, st_ctime=1624852948)

The shutil.copyfileobj() method

The copyfileobj() method is used to copy the file object from source to destination location.

In addition to the source and destination location, we can pass the buffer size, indicating the portion of the file that needs to be copied. The default value for this is 16 KB.

shutil.copyfileobj(fsrc, fdst[, length])

• fsrc : The source file location (path) whose contents need to be copied.
• fdst : The destination file location (path) where the contents from the source file will be pasted.
• length: The buffer size representing the number of bytes kept in memory during the copy process. The default size that the system used is 16KB.

Let us see an example to profit.txt from the report folder to the account folder.

Example:

import shutil

source = r"E:\demos\files\reports\profit.txt"
destination = r"E:\final\reports\profit.txt"
# open source file in read mode
with open(source, "r") as input:
    # open destination path in write mode
    with open(destination, "w") as output:
        # copy file
        shutil.copyfileobj(input, output)
print("Copied")

Copy Files Using OS Module

The Python os module provides functions that are interoperable in different os. We have to use the underlying copy commands specific to the particular operating system. The methods are portable across different operating systems.

Copying files using os.popen() method

The popen() method opens a pipe to execute a command that we pass on the file object. This method opens a file object in the read or the write mode. The read is a default mode.

os.popen(cmd, mode='r', buffering=-1)

• cmd : The command that will be executed in the open file object.
• mode: The mode with which a file can be opened and default mode is r.
• buffering: The number of bytes that will be stored in the buffer. If mentioned as 0 then it will not save any.

The underlying shell command for copying is copy for windows and cp for UNIX environment.

Let us see how to use this method to copy a file called profit.txt from the source folder to the destination folder.

On Unix:

import os

# Setting the current working directory
os.chdir('/Users/pynative/workspace')

# Setting the source and the destination folders
src = os.getcwd() + "/src"
dst = os.getcwd() + "/dst"

os.popen('cp src/Sample.txt dst/Sample.txt')

On Windows:

import os

# Setting the current working directory
os.chdir(r'E:\demos\files_demos\account')

# Setting the source and the destination folders
src = os.getcwd() + "\src"
dst = os.getcwd() + "\dst"

os.popen('copy src\sample.txt dst\sample.txt')

Copying file using os.system() method

The system() method is again used to execute the script in the shell or an OS command. The return value is the exit status of the the command.

• This is implemented by calling the Standard C function system(), and has the same limitations.
• On Unix, the return value is the exit status of the process encoded in the format specified for wait().
• On Windows, the return value is that returned by the system shell after running the command. 

os.system(command)

command : Takes the single argument command which is the underlying shell command that will be executed.

Let us see how to use this method to copy a file called sample.txt from the source folder to the destination folder.

import os

# Setting the current working directory
os.chdir('/Users/pynative/workspace')

# Setting the source and the destination folders
src = os.getcwd() + "/src"
dst = os.getcwd() + "/dst"

# copy file
os.system('cp src/Sample.txt dst/Sample.txt')

Subprocess Module to Copy Files

The subprocess module is used to execute/launch a new subprocess from our application, connect to their input/output/error pipes, and obtain their return codes. This module intends to replace several older modules and functions:

os.system
os.spawn*
os.popen*
popen2.*
commands.*

The subprocess.call() method

This method could be used to execute any command that is passed as an argument to this method and the return value would be end status of the command that was executed.

subprocess.call(args, *, stdin=None, stdout=None, stderr=None, shell=False)

• args : Required for all calls and should be a string, or a sequence of program arguments. The call() method actually runs the command mentioned by the args.
• stdin, stdout and stderr specify the executed program’s standard input, standard output, and standard error respectively. 
• shell: If set to true, the specified command will be executed through the shell.

import os
import subprocess

# Setting the path
os.chdir('/Users/pynative/workspace')

# Assigning source and destination folders
src_folder = os.path.realpath(os.getcwd() + "/src")
dst_folder = os.path.realpath(os.getcwd() + "/dst")

# Assigning source and destination files
src_file = src_folder + "/sample.txt"
dst_file = dst_folder + "/sample.txt"

# Assigning the command with the source and the destination files
cmd = 'cp "%s" "%s"' % (src_file, dst_file)

# Executing the command by calling the subprocess.call()
status = subprocess.call([cmd, src_file, dst_file], shell=True)

# printing the contents of the destination folder
print("Destination folder after copying", os.listdir(dst_folder))

Output

Destination folder after copying ['Sample.txt'] 

Copying Files using subprocess.check_output() method

The check_output() method is again used to execute external commands and capture its output. This is very similar to the call() method with a similar set of arguments.

subprocess.check_output(args, *, stdin=None, stderr=None, shell=False, universal_newlines=False)

universal_newlines : If this value is True then all line endings will be converted to '\n' as described for the universal newlines 'U' mode argument to open().

import os
import subprocess

os.chdir('/Users/pynative/workspace')
# Assigning source and destination folders

src_folder = os.path.realpath(os.getcwd() + "/src")
dst_folder = os.path.realpath(os.getcwd() + "/dst")

# Assigning source and destination files

src_file = src_folder + "/Sample.txt"
dst_file = dst_folder + "/Sample.txt"
cmd = 'cp "%s" "%s"' % (src_file, dst_file)

status = subprocess.check_output([cmd, src_file, dst_file], shell=True)

# printing the contents of the destination folder
print("Destination folder after copying", os.listdir(dst_folder))

Output

Destination folder after copying ['Sample.txt']