SSH command execution and SFTP file transfer using paramiko in Python

11 Aug 2022 - tsp
Last update 21 Sep 2022
Reading time 6 mins

Disclaimer: This blog entry is more of a note of some simple recipes used over and over again.

Introduction

Sometimes one has some Python scripts running and wants to transfer data to or fetch data from a remote machine - or simply execute commands. The best way to run commands on remote machines usually is SSH, the best ways for file transfer SFTP, SCP or simply copying to an NFS share (or of course using WebDAV / DeltaV). A simple method to realize this is the usage of the excellent paramiko library - this is a Python implementation of the SSHv2 protocol that builds on top of the cryptography package for native cryptography primitives.

This library is pretty flexible - this short blog entry will only show the most basic usage and also assumes that one wants to use public key based authentication for the remote user. As soon as one has grasped the basic ideas one can figure out everything necessary from the excellent documentation.

Installation

One can directly install paramiko from PyPi:

pip install paramiko

Importing the library, loading the private key file

First as usual one has to import the library:

import paramiko

Then one can use the RSAKey.from_private_key_file method to load the private key part for authentication:

try:
    sshkey = paramiko.RSAKey.from_private_key_file("/path/to/keyfile")
except IOError:
    # The file has not been found or any other kind of I/O error has happened
    pass
except PasswordRequiredException:
    # No password has been supplied by the keyfile is password protected
    pass
except SSHException
    # Keyfile is invalid
    pass

To supply a password one can add the password argument to the from_private_key_file method. Please keep in mind that handling passwords usually should be done with care - using separate memory locations that are not pageable, etc. - this is pretty hard to do correct from Pythons environment.

Executing remote commands

To execute remote commands one just has to:

Host key handling tells the client how to react when one sees an unknown or changed remote host key. The typical policies are:

# First load the SSHKey - see above for details:
sshkey = paramiko.RSAKey.from_private_key_file("/path/to/keyfile")

# Create the SSH client and setup handling of new host keys
sshClient = paramiko.SSHClient()
sshClient.set_missing_host_key_policy(paramiko.AutoAddPolicy())

# Connect to the remote host
sshClient.connect("203.0.113.1", username = "example", pkey = sshkey)

To execute commands one uses the exec_command method that returns a tuple of the standard pipes:

stdin, stdout, stderr = sshClient.exec_command(command)

To dump all information sent by a client and simply terminate afterwards one can dump information from stdout and invoke all destructors on the pipes. Note that this is required!:

stdin, stdout, stderr = sshClient.exec_command("uname -a")
print(stdout.read())
del stdin, stdout, stderr

In the end one has to close the client:

sshClient.close()

File transfer via SFTP

To transfer files one can use the sftp subsystem of SSH. Paramiko provides a full blown implementation of the SFTP protocol including directory traversal, permission management, reading and writing files without storing them locally, etc. The following example will limit itself to uploading and downloading of files as well as deleting and listing.

Bootstrapping of the public key, client instantiating and connecting works the same as before:

# First load the SSHKey - see above for details:
sshkey = paramiko.RSAKey.from_private_key_file("/path/to/keyfile")

# Create the SSH client and setup handling of new host keys
sshClient = paramiko.SSHClient()
sshClient.set_missing_host_key_policy(paramiko.AutoAddPolicy())

# Connect to the remote host
sshClient.connect("203.0.113.1", username = "example", pkey = sshkey)

Then one has to open the SFTP client:

sftpClient = sshClient.open_sftp()

Upload and download

Both methods allow one to set a callback = callable that receives two integer parameters callback(bytesTransferred, bytesTotal). One could use this to provide status updates for example:

def showTransferStatus(bytesTransferred, bytesTotal):
    print(f"{bytesTransferred} out of {bytesTotal} bytes transferred")

sftpClient.put("/local/source", "/remote/destination", callback = showTransferStatus)

Delete

To delete a file one can simply call the remove function:

sftpClient.remove("/path/remote")

Listing files

There are a variety of methods to list files on the remote machine using the SFTP client. The easiest for small directories is listdir(path = '.'). This method returns a simple Python list containing all filenames and directory names on the remote side. It does not contain entries for . and .. though.

print(sftpClient.listdir())

In case one also wants to receive the attributes one might use listdir_attr(path = '.'). This returns an wrapper containing additional information per entry:

A more advanced method to list files uses an iterator / generator pattern. This method is called listdir_iter(path = '.', read_aheads = 50). This method can be used in a typical iterator pattern like a for each loop:

for f in sftpClient.listdir_iter():
	print(f.filename)

The objects iterated over have the same structure as the result of listdir_attr.

Shutting down

In the end the SFTP client as well as the SSH client has to be shut down:

sftpClient.close()
sshClient.close()

This article is tagged: Python, Basics, Programming


Data protection policy

Dipl.-Ing. Thomas Spielauer, Wien (webcomplains389t48957@tspi.at)

This webpage is also available via TOR at http://jugujbrirx3irwyx.onion/

Valid HTML 4.01 Strict Powered by FreeBSD IPv6 support