In this Python tutorial, you’ll learn how to use various functions available in the os, shutil, and subprocess modules to copy files and folders from one location to another.
After reading this article, you’ll learn: –
- How to copy files in Python using shutil module’s
copy()
,copy2()
,copyfiles()
,copyfileobj()
methods - The OS and subprocess module to copy files using the underlying operating system’s shell command (
copy
in Windows orcp
in UNIX.) - How to copy all files from a directory
- Copy an entire directory recursively
Table of contents
Steps to Copy a File in Python
Python provides strong support for file handling. We can copy single and multiple files using different methods and the most commonly used one is the shutil.copy()
method. The below steps show how to copy a file from one folder to another.
- Find the path of a file
We can copy a file using both relative path and absolute path. The path is the location of the file on the disk.
An absolute path contains the complete directory list required to locate the file. For example,/home/Pynative/samples.txt
is an absolute path to discover the samples.txt. - Use the shutil.copy() function
The shutil module offers several functions to perform high-level operations on files and collections of files. The
copy()
function in this module is used to copy files from one directory to another.
First, import the shutil module and Pass a source file path and destination directory path to thecopy(src, dst)
function. - Use the os.listdir() and shutil copy() function to copy all files
Suppose you want to copy all files from one directory to another, then use the
os.listdir()
function to list all files of a source folder, then iterate a list using a for loop and copy each file using thecopy()
function. - Use copytree() function to copy entire directory
The
shutil.copytree(src, dst)
recursively copy an entire directory tree rooted atsrc
to a directory nameddst
and return the destination directory
Example: Copy Single File
In this example, we are copying the profit.txt file from the report folder to the account folder.
import shutil
src_path = r"E:\demos\files\report\profit.txt"
dst_path = r"E:\demos\files\account\profit.txt"
shutil.copy(src_path, dst_path)
print('Copied')
Copy All Files From A Directory
Sometimes we want to copy all files from one directory to another. Follow the below steps to copy all files from a directory.
- Store the source and destination directory path into two variables
- Get the list of all files present in the source folder using the
os.listdir()
function. It returns a list containing the names of the files and folders in the given directory. - Iterate over the list using a for loop to get the individual filenames
- In each iteration, concatenate the current file name with the source folder path
- Now use the
shutil.copy()
method to copy the current file to the destination folder path.
Example:
import os
import shutil
source_folder = r"E:\demos\files\reports\\"
destination_folder = r"E:\demos\files\account\\"
# fetch all files
for file_name in os.listdir(source_folder):
# construct full file path
source = source_folder + file_name
destination = destination_folder + file_name
# copy only files
if os.path.isfile(source):
shutil.copy(source, destination)
print('copied', file_name)
Output:
copied expense.txt copied profit.txt copied revenue.txt
Copy Entire Directory
Sometimes we need to copy an entire folder, including all files and subfolders contained in it. Use the copytree()
method of a shutil module to copy the directory recursively.
shutil.copytree(src, dst, symlinks=False, ignore=None, copy_function=copy2, ignore_dangling_symlinks=False, dirs_exist_ok=False
- This method recursively copy an entire directory tree rooted at
src
to a directory nameddst
and return the destination directory. - The
dirs_exist_ok
dictates whether to raise an exception in casedst
or any missing parent directory already exists. - Use the
copy_function
parameter to pass any of the four functions the shutil module provides to copy files.
Example: Let’s to how to copy the report directory and all its content into an account directory.
import shutil
source_dir = r"E:\demos\files\reports"
destination_dir = r"E:\demos\files\account"
shutil.copytree(source_dir, destination_dir)
Shutil Module to Copy files
Python shutil module offers many high-end functions to perform copy and removal of files.
These functions provide an optimized way to copy files and thereby save time in performing the unnecessary task of opening, reading, writing, and closing the files when there is no processing required in that file.
The shutil module offers the following four functions to copy files.
Function | Description |
---|---|
shutil.copyfileobj() | Copy the file object from a source to the destination. (Need to open both source and destination path) |
shutil.copyfile() | Copy the contents of one file to another file. The metadata of the file will not be copied. |
shutil.copy() | copy the contents of the source file to the destination file along with metadata. |
shutil.copy2() | copy the additional metadata namely timestamps of the source file to the destination |
To decide which function to use for copying a file, we have to consider the following points.
- For example, If your application uses the shared resources, you should copy a file in blocking mode instead of asynchronously.
- Platform portability. If your application runs on a different operating system, you must write a code that will run on a different OS without any issue.
- The function is resource-intensive as some copy functions involve opening the file, reading the contents, writing and closing the file. so choose as per your need
- Do you want to copy metadata along with file content?
The shutil.copyfile()
method
The copyfile() method is used to copy the contents of one file to another file. The metadata of the file will not be copied.
shutil.copyfile(src, dst, *, follow_symlinks=True)
fsrc
: The source file location (path) whose contents need to be copied.fdst
: The destination file location (path) where the contents from the source file will be pasted.follow_symlinks
: The default value for this is true. If it is false and the source file location is a symbolic link then a new symbolic link will be created instead of copying from the file.
Note: The metadata of the file will not be copied while using this method.
Example:
import shutil
src_folder = r"E:\demos\files\reports"
dst_folder = r"E:\demos\files\account"
# file names
src_file = src_folder + "\profit.txt"
dst_file = dst_folder + "\profit.txt"
shutil.copyfile(src_file, dst_file)
print('Copied')
Our code copied the ‘profit.txt’ file in the destination directory. Here is a list of the files present in the destination directory:
- profit.txt
- revenue.txt
Use the os.listdir(dst_folder)
function to list all files present in the destination directory to verify the result.
The following are the important points
- Both the source and destination should be a file. If the destination is a directory then
IsADirectoryError
will be raised. - If the source and destination point to the same file then the
SameFileError
will be raised. - If the destination exists but with a different name compared to the source file then the contents of the destination file will be overwritten.
To avoid the above errors always wrap your code in the try-except block. Refer the following example.
import os
import shutil
from shutil import SameFileError
src_folder = r"E:\demos\files\reports"
dst_folder = r"E:\demos\files\account"
# printing the contents of the destination folder
print("Destination folder before copying::", os.listdir(dst_folder))
# file names
src_file = src_folder + "\profit.txt"
dst_file = dst_folder + "\profit.txt"
try:
# copy file
shutil.copyfile(src_file, dst_file)
# destination folder after copying
print("Destination after copying", os.listdir(dst_folder))
except SameFileError:
print("We are trying to copy the same File")
except IsADirectoryError:
print("The destination is a directory")
Output
Destination folder before copying:: ['expense.txt', 'revenue.txt'] Destination after copying ['expense.txt', 'profit.txt', 'revenue.txt']
The shutil.copy()
method
The shutil
module has one more method to copy the file called copy()
method which will copy the contents of the source file to the destination file.
shutil.copy(src, dst, *, follow_symlinks=True)
Similar to the copyfile()
method this method will throw the ‘SameFileError
‘ if the source and destination point to the same file.
Consider the following example where we try to copy the file ‘profit.txt’ from the source src
to the destination dst
directory.
import shutil
src_path = r"E:\demos\files\reports\profit.txt"
dst_path = r"E:\demos\files_demos\account\profit.txt"
shutil.copy(src_path, dst_path)
print('Copied')
Note: If the source file is not found in the location then the copy() method can’t be executed and an IOError
will be raised. If the destination folder is not writable then ‘PermissionError
‘ will be raised.
copy() vs copyfiles()
While the copy()
and copyfile()
methods in the shutil module work the same way in copying files from one location to another there are some significant differences between them.
If the destination path is ‘file’ or ‘directory’ then the copy()
method will create a file in the name of the source file.
- The
copy()
method saves the additional metadata information about the permission modes from the source to the destination file. Thecopyfile()
doesn’t save any metadata information. - As the
copy()
method makes one additional call to save the permission modes; it is slower when compared to thecopyfile
() method.
The copy()
method calls the copyfile()
and copymode()
inside its implementation internally. But the copyfile
() only calls the copyfileobj()
inside its implementation
The shutil.copy2()
method
In shutil
there is one more method called copy2()
which could be used to copy the file from one location to another.
shutil.copy2(src, dst, *, follow_symlinks=True)
While the copy()
and the copy2()
methods both copy the contents of the file along with the metadata there are two significant differences between these two methods.
- The
copy2()
method can be used to copy the additional metadata namely timestamps of the source to the destination. This is in addition to the permission modes which thecopy()
method saves. - The
copy2()
method calls thecopystat()
method to get the timestamp information while thecopy()
method saves the permission modes from thecopymode()
The scenarios that this method throws the ‘SameFileError’ are similar to the copy() method.We can see how to use this method to copy the file named ‘Sample.txt’ from the source to the destination.
import os
import shutil
src_path = r"E:\demos\files\reports\profit.txt"
dst_path = r"E:\demos\files\account\profit.txt"
# metadata of source file
metadata_src = os.stat(src_path)
print("Metadata source:", metadata_src)
# copy file
shutil.copy2(src_path, dst_path)
print('Copied')
metadata_dst = os.stat(dst_path)
print("Metadata destination:", metadata_dst)
Ouput
Metadata source: os.stat_result(st_mode=33206, st_ino=4222124650735236, st_dev=3202242018, st_nlink=1, st_uid=0, st_gid=0, st_size=0, st_atime=1624768574, st_mtime=1623330684, st_ctime=1624768574) Copied Metadata destination: os.stat_result(st_mode=33206, st_ino=5348024557577894, st_dev=3202242018, st_nlink=1, st_uid=0, st_gid=0, st_size=0, st_atime=1624768574, st_mtime=1623330684, st_ctime=1624852948)
The shutil.copyfileobj()
method
The copyfileobj()
method is used to copy the file object from source to destination location.
In addition to the source and destination location, we can pass the buffer size, indicating the portion of the file that needs to be copied. The default value for this is 16 KB.
shutil.copyfileobj(fsrc, fdst[, length])
fsrc
: The source file location (path) whose contents need to be copied.fdst
: The destination file location (path) where the contents from the source file will be pasted.length
: The buffer size representing the number of bytes kept in memory during the copy process. The default size that the system used is 16KB.
Let us see an example to profit.txt from the report folder to the account folder.
Example:
import shutil
source = r"E:\demos\files\reports\profit.txt"
destination = r"E:\final\reports\profit.txt"
# open source file in read mode
with open(source, "r") as input:
# open destination path in write mode
with open(destination, "w") as output:
# copy file
shutil.copyfileobj(input, output)
print("Copied")
Copy Files Using OS Module
The Python os module provides functions that are interoperable in different os. We have to use the underlying copy commands specific to the particular operating system. The methods are portable across different operating systems.
Copying files using os.popen()
method
The popen()
method opens a pipe to execute a command that we pass on the file object. This method opens a file object in the read or the write mode. The read is a default mode.
os.popen(cmd, mode='r', buffering=-1)
cmd
: The command that will be executed in the open file object.mode
: The mode with which a file can be opened and default mode isr
.buffering
: The number of bytes that will be stored in the buffer. If mentioned as 0 then it will not save any.
The underlying shell command for copying is copy
for windows and cp
for UNIX environment.
Let us see how to use this method to copy a file called profit.txt from the source folder to the destination folder.
On Unix:
import os
# Setting the current working directory
os.chdir('/Users/pynative/workspace')
# Setting the source and the destination folders
src = os.getcwd() + "/src"
dst = os.getcwd() + "/dst"
os.popen('cp src/Sample.txt dst/Sample.txt')
On Windows:
import os
# Setting the current working directory
os.chdir(r'E:\demos\files_demos\account')
# Setting the source and the destination folders
src = os.getcwd() + "\src"
dst = os.getcwd() + "\dst"
os.popen('copy src\sample.txt dst\sample.txt')
Copying file using os.system()
method
The system() method is again used to execute the script in the shell or an OS command. The return value is the exit status of the command.
- This is implemented by calling the Standard C function
system()
, and has the same limitations. - On Unix, the return value is the exit status of the process encoded in the format specified for
wait()
. - On Windows, the return value is that returned by the system shell after running the command.
os.system(command)
command
: Takes the single argument command which is the underlying shell command that will be executed.
Let us see how to use this method to copy a file called sample.txt from the source folder to the destination folder.
import os
# Setting the current working directory
os.chdir('/Users/pynative/workspace')
# Setting the source and the destination folders
src = os.getcwd() + "/src"
dst = os.getcwd() + "/dst"
# copy file
os.system('cp src/Sample.txt dst/Sample.txt')
Subprocess Module to Copy Files
The subprocess module is used to execute/launch a new subprocess from our application, connect to their input/output/error pipes, and obtain their return codes. This module intends to replace several older modules and functions:
os.system
os.spawn*
os.popen*
popen2.*
commands.*
The subprocess.call()
method
This method could be used to execute any command that is passed as an argument to this method and the return value would be end status of the command that was executed.
subprocess.call(args, *, stdin=None, stdout=None, stderr=None, shell=False)
args
: Required for all calls and should be a string, or a sequence of program arguments. The call() method actually runs the command mentioned by the args.stdin
,stdout
andstderr
specify the executed program’s standard input, standard output, and standard error respectively.shell
: If set to true, the specified command will be executed through the shell.
import os
import subprocess
# Setting the path
os.chdir('/Users/pynative/workspace')
# Assigning source and destination folders
src_folder = os.path.realpath(os.getcwd() + "/src")
dst_folder = os.path.realpath(os.getcwd() + "/dst")
# Assigning source and destination files
src_file = src_folder + "/sample.txt"
dst_file = dst_folder + "/sample.txt"
# Assigning the command with the source and the destination files
cmd = 'cp "%s" "%s"' % (src_file, dst_file)
# Executing the command by calling the subprocess.call()
status = subprocess.call([cmd, src_file, dst_file], shell=True)
# printing the contents of the destination folder
print("Destination folder after copying", os.listdir(dst_folder))
Output
Destination folder after copying ['Sample.txt']
Copying Files using subprocess.check_output()
method
The check_output()
method is again used to execute external commands and capture its output. This is very similar to the call()
method with a similar set of arguments.
subprocess.check_output(args, *, stdin=None, stderr=None, shell=False, universal_newlines=False)
universal_newlines
: If this value is True
then all line endings will be converted to '\n'
as described for the universal newlines 'U'
mode argument to open()
.
import os
import subprocess
os.chdir('/Users/pynative/workspace')
# Assigning source and destination folders
src_folder = os.path.realpath(os.getcwd() + "/src")
dst_folder = os.path.realpath(os.getcwd() + "/dst")
# Assigning source and destination files
src_file = src_folder + "/Sample.txt"
dst_file = dst_folder + "/Sample.txt"
cmd = 'cp "%s" "%s"' % (src_file, dst_file)
status = subprocess.check_output([cmd, src_file, dst_file], shell=True)
# printing the contents of the destination folder
print("Destination folder after copying", os.listdir(dst_folder))
Output
Destination folder after copying ['Sample.txt']