bearsadventures.co.uk

Linux-SFTPServer

Sometimes you need to transfer files between two computers. This can take the form of either transferring files from you local computer to your remote server or from your remote computer back to your local computer. Before we had more mature continuous deployment tools like Azure Pipelines, Travis CI, or GitHub Actions, we often used a tool called ftp (file transfer protocol). While it worked and it was a relatively simple and straightforward of doing things, we moved onto to more reliable tools. However ftp evolved into secure ftp which we still use today.

How to 1

How to set up sftp

Okay, buckle up. This is your biggest challenge yet.

Just kidding, you’re already done. One of the best things about sftp is that it works 100% over the same ways that ssh does so if you’ve set up ssh you’ve inherently set up sftp too (it possible to set up one and not the other but you have change some options.) This was a welcome departure from ftp which has its own setup process and ports that had to be managed.

Okay so using the VMs we set up in the previous section, let’s sftp into secondary from primary. The sftp interactive shell is similar to a less friendly and stripped down version of bash. The thing to keep in mind is you’re in two directories at the same time as opposed to normal bash when you’re just in one. With sftp, you have a local context and a remote context. In order to run a local command, tack an l to the start of whatever command you’re running, like lls, lpwd, or lcd. When you want to do it on the remote machine, just run the commands like normal, like ls, pwd, or cd.

sftp Craig@<the same ip from the previous step>
lpwd # ubuntu's local home directory
pwd # Craig's remote home directory
lls # the list of files in ubuntu's home directory
ls # the list of files in Craig's home directory
help # see all the commands you can do

Feel free to navigate around. I tend to not spend too much time inside sftp because it’s easier do everything in ssh except file transfers. The two key commands you need to know here are get and put. It’s from the perspective of your local computer so in this case we’re connecting from primary to secondary, so getting something will download it from secondary and putting someting will upload it.

If you have some files already laying around go ahead and toy with it. This is a great thing to use with tar as well so you can bundle up packages of files before sending them to a different computer.

So now we want to upload a file. But hey, you may not have a file to upload right now. It’d be great if we could run a quick command on our local computer. And you can! Just prefix it with ! and you can quickly run a touch or a tar.

!touch file-to-put.txt

Obviously I wouldn’t encourage you to do a lot here; if you need to just disconnect and then reconnect later, but it’s helpful for quick things you need to do.

put file-to-put.txt putted-file.txt # second argument is optional, if you omit it'll just use the same name
get putted-file.txt gotten-file.txt # same thing, second one is optional

Here we went through the very fruitful exercise of putting a file on a server, renaming it and then downloading it again. But now you can see how you can download and upload files.

And that’s it! sftp can do a few more things like make directories and chmod/chown stuff but I’ll leave you to poke and prod that as you need to.

How to 2

1. Overview

The Secure File Transfer Protocol (SFTP) is a tool for safely transferring files over the Internet. The SFTP tool can be accessed on a Linux system using the sftp command. It allows us to manage files remotely.

In this tutorial, we’ll learn how to use the sftp command effectively. We’ll cover the basics of the command and common options you can use with it.

2. The sftp Command

The sftp command is part of the OpenSSH package. It’s our go-to tool for secure file transfers on Linux. Unlike regular FTP, sftp uses the SSH protocol to encrypt the connection and the data we send. This helps to protect our files from prying eyes.

Now, let’s take a look at the basic way to use the command:

sftp [options] [user@]host

Let’s break this down:

  • options: are extra settings we can add to customize how sftp works
  • user@host: tells sftp where to connect. It’s made up of user – username on the remote computer and host that specifies the address of the remote computer

When we connect to sftp, it opens an interactive session. Here, we can type commands to manage files on the remote system.

Beyond just transferring files, sftp allows us to perform various tasks on the remote machine. We can list files and directories, delete files, create new folders, and more.

3. Common sftp Command Options

To make the most of sftp, we should understand some of the commonly used options that can enhance our experience with the tool. We need to add these options before the destination address when running the sftp command.

3.1. Connection and Authentication Options

The sftp command offers several options to control how we connect to remote servers and authenticate ourselves. Let’s learn about some of these options:

  • -P [port]: allows us to specify the port to connect to on the remote host if it uses a port other than the default port (22) for SSH connections
  • -i [identity_file]: is helpful when working with servers that require key-based authentication (a more secure way to log in than passwords)
  • -o [ssh_option]: gives us even more control over how sftp connects using SSH. We can use this option to set things like proxy commands, host key checking, and where to look for known host files

These options ensure we can use sftp to flexibly connect to different servers with different configurations and security requirements.

3.2. File Transfer Options

sftp also offers us several options to help tweak the actual file transfer process. Now, let’s see some of them:

  • -a: allows us to pick up from where we left off after an interruption
  • -p: comes in handy when we need to keep the exact timestamps (the last time we modified or accessed the file) and permissions of the files we’re transferring
  • -R [num_requests]: allows us to control how many file transfer requests that can happen at the same time
  • -b [batchfile]: is a time-saver when we need to transfer multiple files or execute a series of commands. Instead of typing each command manually, we can create a text file (batchfile) containing the commands we want to run, and sftp will execute them automatically
  • -l [limit]: is helpful in situations where we need to set a limit on how fast sftp can transfer data. We can specify the limit in Kbits/s (kilobytes per second). This way, we don’t have to worry about sftp using up all our bandwidth

With these options, we have control over how files are handled during transfer. This makes sftp adaptable to different needs and situations.

4. Common sftp Command Examples

We’ve learned a few options we can use with sftp. Now, let’s see some of these options in action. These scenarios will cover connecting to a server, transferring files, and managing files on the remote system.

4.1. Connecting to an SFTP Server

Connecting to an SFTP server is the first step to transferring or managing files remotely. Let’s look at a few ways we can establish a secure connection.

First, let’s make a basic connection using the sftp command and a destination:

$ sftp [email protected]

The above command initiates a connection to the SFTP server on the host, 191.22.235.149, using the username baeldung. We’ll typically be prompted to enter a password unless we’ve set up SSH keys for authentication.

Also, we can specify a non-default port when connecting to our SFTP server:

$ sftp -P 22222 [email protected]

Here, we used the -P option to specify a custom port. This is helpful when our server isn’t running on the default port 22.

In addition, when we don’t want to connect to our server using a password, we can use a private key:

$ sftp -i ~/.ssh/id_rsa [email protected]

This improves security and convenience. In the example above, we specify the path to our private key file (often stored in ~/.ssh/id_rsa) using the -i option. This eliminates the need to type our password each time we want to connect.

4.2. Transferring Files

Transferring files with the sftp command is easy. It gets even better with options that help us adjust the process.

Let’s say we want to upload a large file, but our internet isn’t stable. We can use the -a option with sftp:

$ sftp -a [email protected]
sftp> put datafile.txt /remote/directory/remote_datafile.txt

With the -a option, if our internet connection breaks at any point, we don’t have to start the upload from scratch. Instead, sftp will pick up right where it left off.

In the interactive sftp session above, the put command uploads datafile.txt from our local machine to the /remote/directory/ on the server. Also, it renames the file to remote_datafile.txt on the server.

Furthermore, we can preserve file attributes such as timestamps and permissions from the original files in the transferred files using the -p option:

$ sftp -p [email protected]
sftp> put datafile.txt /remote/directory/remote_datafile.txt

With this option, our remote_datafile.txt file retains the attributes of the original datafile.txt file. This ensures consistency in the file attributes.

Next, if we have a fast internet connection, we can speed up transfers by allowing sftp to handle multiple file requests at the same time using the -R option:

$ sftp -R 10 [email protected] sftp> put datafile.txt /remote/directory/remote_datafile.txt

The -R option in this example tells sftp to allow up to 10 simultaneous requests. This will make our file transfer much faster.

4.3. Managing Remote Files

Once connected to an SFTP server, we can easily manage files directly on the remote system. This can be quite helpful when we need to tidy things up, rename files, or delete unwanted files without downloading them first.

For example, we can change the directory using the cd command:

$ sftp [email protected]
sftp> cd /new/directory

This is like changing folders on our local computer. The cd command allows us to navigate to a different directory on the remote server. In the above example, we’re moving to a folder called /new/directory/.

Also, we can list files in a remote directory using ls:

$ sftp [email protected]
sftp> ls

Just like on our local machine, the ls command displays a list of all the files in the current or specified directory. This is helpful when we want to see what’s in a particular folder on the remote server.

In addition, when we need to get rid of files we no longer need on the server, the rm command comes to the rescue:

$ sftp [email protected]
sftp> rm remote_datafile.txt

In this case, the command deletes the file remote_datafile.txt. However, we have to be careful, as files deleted using sftp are gone for good.

Lastly, we can create a new directory using the mkdir command:

$ sftp [email protected]
sftp> mkdir new_directory

The command creates a new folder called new_directory on the server.

5. Conclusion

In this article, we explored the Linux sftp command. We began by establishing a secure connection with various authentication methods. This is to ensure the privacy and integrity of our data during transit.

Furthermore, we took a walk through the range of options available for customizing file transfers, including resuming interrupted transfers, preserving file attributes, and more. We also got our hands dirty with some practical examples to help us understand how to use these options.

By using the power of sftp, we can confidently transfer sensitive data, knowing it’s protected by a robust encryption system.