In Linux which of the following resources are not available through file descriptors

If a descriptor is no longer used by a program and is not closed explicitly, its number cannot be reused (which is problematic in itself, see Dealing with the select Limit), and the kernel resources are not freed. Therefore, it is important to close all descriptors at the earliest point in time possible, but not earlier.

Error Handling during Descriptor Close

The close system call is always successful in the sense that the passed file descriptor is never valid after the function has been called. However, close still can return an error, for example if there was a file system failure. But this error is not very useful because the absence of an error does not mean that all caches have been emptied and previous writes have been made durable. Programs which need such guarantees must open files with O_SYNC or use fsync or fdatasync, and may also have to fsync the directory containing the file.

Closing Descriptors and Race Conditions

Unlike process IDs, which are recycle only gradually, the kernel always allocates the lowest unused file descriptor when a new descriptor is created. This means that in a multi-threaded program which constantly opens and closes file descriptors, descriptors are reused very quickly. Unless descriptor closing and other operations on the same file descriptor are synchronized (typically, using a mutex), there will be race conditions and I/O operations will be applied to the wrong file descriptor.

Sometimes, it is necessary to close a file descriptor concurrently, while another thread might be about to use it in a system call. In order to support this, a program needs to create a single special file descriptor, one on which all I/O operations fail. One way to achieve this is to use socketpair, close one of the descriptors, and call shutdown(fd, SHUTRDWR) on the other.

When a descriptor is closed concurrently, the program does not call close on the descriptor. Instead it program uses dup2 to replace the descriptor to be closed with the dummy descriptor created earlier. This way, the kernel will not reuse the descriptor, but it will carry out all other steps associated with calling a descriptor (for instance, if the descriptor refers to a stream socket, the peer will be notified).

This is just a sketch, and many details are missing. Additional data structures are needed to determine when it is safe to really close the descriptor, and proper locking is required for that.

Lingering State after Close

By default, closing a stream socket returns immediately, and the kernel will try to send the data in the background. This means that it is impossible to implement accurate accounting of network-related resource utilization from userspace.

The SO_LINGER socket option alters the behavior of close, so that it will return only after the lingering data has been processed, either by sending it to the peer successfully, or by discarding it after the configured timeout. However, there is no interface which could perform this operation in the background, so a separate userspace thread is needed for each close call, causing scalability issues.

Currently, there is no application-level countermeasure which applies universally. Mitigation is possible with iptables (the connlimit match type in particular) and specialized filtering devices for denial-of-service network traffic.

These problems are not related to the TIME_WAIT state commonly seen in netstat output. The kernel automatically expires such sockets if necessary.

Let’s talk about Linux file descriptors and how to investigate a malicious process using them.

What Is a File Descriptor?

Since the inception of Unix (on which Linux is based), everything is a file. We can quibble about the specifics, but mostly this is true and for our purposes is all we need to know. A file descriptor is a handle used by the operating system to access a file or other I/O mechanism. It is the same as a phone number you dial to get a specific person, except on Linux the number will give you a pointer to a file.

We’re simplifying this for our discussion, but just know that basically all processes on Linux that are running are going to have some kind of file descriptor open. These file descriptors can be any legal file or resource on Linux such as:

• Files
• Network sockets
• Named pipes
• Devices

On Linux the most basic file descriptors you’ll see open by most processes will be stdin, stdout and stderr. These allow the process to communicate back to the terminal and take data input (stdin), output data to the terminal (stdout) and pass out errors (stderr). But more than this, the file descriptors will also show what files, sockets, etc. the process may be talking to when running. Often live malware will write data to the disk it is stealing or using for other reasons (such as data logs).

The key point here is if the process is running you can quickly see what files it has open and can use that information to determine if it is malicious and what it may be doing.

Show Me the File Descriptors Already

To find the open file descriptors of a process, we will go to our old friend the /proc file system. The data we want is here:

/proc/<PID>/fd

The basic format for listing the open file descriptors of a process is simply:

ls -al /proc/<PID>/fd

Once you do this you’ll see what files the process has open and if any are interesting.

Suspicious Linux Process Example

Below we see a suspicious process called tmpwrk and it has a particular interest in a file under the /tmp directory. We can use this information to view the contents of that file and determine what may be going on.

We look under the /proc/<PID>/fd directory and do a simple ls command to see what the process has open:

We see a file and a socket open. The file has a very strange name of /tmp/.data and we want to see what it is. We can use standard file tools on the file link itself or the file it links to directly.

Below we cat out the file and see it holds some secret data. This can help confirm that the binary is in fact malicious. We can view the file, copy it, run hashes on it or anything else. It’s a standard file.

What About the Open Socket?

The open socket can also be investigated by using the number in [brackets] which is the inode. You can run it through netstat in a variety of ways. Andrew Case contributed in a tweet this command:

netstat -eepan | grep <INODE>

Where INODE is again the number in brackets from above. This will show you the kind of socket the process is using (e.g. network).

What About lsof?

lsof is a tool that shows all open files on a system. If not installed, you will have to use the methods above. You can use lsof with the following:

lsof -p <PID>

lsof -c <command name>

The lsof command will show all open files in one easy to use way.

Why Not Use lsof?

There are two reasons why using lsof may not be the first thing to do when looking at a suspicious process:

1. It may not be installed and you don’t want to alter a compromised machine by loading new packages.
2. The lsof command is not commonly called and could alert malware or an intruder about investigators present.

The last point may be a bit academic, but the simpler ls and cat commands are very common and hard to tell why they may be used. lsof on the other hand is pretty specific and there is always the chance it is being targeted by malware to avoid detection or worse. This is a small risk, but it is there.

Let the Malware Help You

The biggest takeaway from this is a malicious process is happy to tell you what it is interested in if you take the time to look at what files it has open. You can do this without the risk of attaching debuggers and it is far safer and faster.

The second takeaway is to never go in and kill a suspicious process on Linux until live process forensics have determined what it may be doing. If you go in guns blazing you will not be able to see what the malware may be up to, what files it has open, who it is talking to and other useful pieces of data. Put away the kill -9 and take your time when investigating a Linux process. It will save you lots of effort.

What are the file descriptors in Linux?

What resources are available through file descriptors?

What are the 3 file descriptors in Linux?

Is stdout available through file descriptors in Linux?