File Descriptor

A File Descriptor (FD) is a non-negative integer that uniquely identifies an open file within a process. It serves as an index into the file descriptor table, which points to the file table entries in the kernel.

• When a process opens an existing file or creates a new one, the kernel returns a file descriptor to the process.
• System calls like read, write, close, and lseek take the file descriptor as an argument to perform operations on the specific file.

Standard File Descriptors

When a program starts, the operating system automatically opens three file descriptors:

1. Standard Input (stdin): FD value 0. Used for reading input (usually keyboard).
2. Standard Output (stdout): FD value 1. Used for writing output (usually terminal screen).
3. Standard Error (stderr): FD value 2. Used for writing error messages (usually terminal screen).

open() System Call

The open() system call is used to establish a connection between a file and a file descriptor.

Syntax:
c

include <fcntl.h>

int open(const char *pathname, int flags, mode_t mode);

Arguments:

• pathname: The path to the file.
• flags: Must include one of the access modes (O_RDONLY, O_WRONLY, O_RDWR) and can be bitwise OR'd with creation flags.
• mode: Required only when creating a new file (specifies permissions).

Flags Explanation:

• O_RDONLY: Open the file for read-only access.
• O_WRONLY: Open the file for write-only access.
• O_RDWR: Open the file for both reading and writing.
• O_CREAT: Create the file if it does not exist. If this flag is used, the mode argument must be supplied to set file permissions.

The read() system call attempts to read a specified number of bytes from a file descriptor into a buffer.

Syntax:
ssize_t read(int fd, void *buf, size_t count);

Return Value Analysis

1. Positive Integer ():

   • Indicates the number of bytes successfully read.
   • This may be less than the requested count if fewer bytes are available (e.g., closer to the end of the file or reading from a pipe).
   • The file position is advanced by this number.

2. Zero ($0$):

   • Indicates End of File (EOF).
   • It means the file position is at or past the end of the file, and no data was read.

3. Negative One ():

   • Indicates an error occurred (e.g., file descriptor is invalid, reading a directory, or hardware error).
   • The global variable errno is set to indicate the specific error.

C Program to Copy File

c

include <fcntl.h>

include <unistd.h>

include <stdio.h>

include <stdlib.h>

define BUFFER_SIZE 1024

int main() {
int sourceFd, destFd;
ssize_t bytesRead, bytesWritten;
char buffer[BUFFER_SIZE];

// 1. Open source file for reading
sourceFd = open("source.txt", O_RDONLY);
if (sourceFd == -1) {
    perror("Error opening source file");
    exit(1);
}

// 2. Open destination file (Create if not exists, Truncate if exists)
// Permissions 0644: rw-r--r--
destFd = open("destination.txt", O_WRONLY | O_CREAT | O_TRUNC, 0644);
if (destFd == -1) {
    perror("Error opening destination file");
    close(sourceFd);
    exit(1);
}

// 3. Read loop
while ((bytesRead = read(sourceFd, buffer, BUFFER_SIZE)) > 0) {
    bytesWritten = write(destFd, buffer, bytesRead);
    if (bytesWritten != bytesRead) {
        perror("Write error");
        close(sourceFd);
        close(destFd);
        exit(1);
    }
}

if (bytesRead == -1) {
    perror("Read error");
}

// 4. Close descriptors
close(sourceFd);
close(destFd);

printf("File copied successfully.\n");
return 0;

}

lseek() System Call

The lseek() system call repositions the file offset of the open file description associated with the file descriptor.

Syntax:
off_t lseek(int fd, off_t offset, int whence);

• fd: File descriptor.
• offset: Number of bytes to move (can be negative).
• whence: The reference point from which the offset is applied.

whence Arguments

1. SEEK_SET:

   • The file offset is set to offset bytes.
   • Essentially, it sets the position relative to the beginning of the file.
   • Example: lseek(fd, 0, SEEK_SET) moves to the start of the file.

2. SEEK_CUR:

   • The file offset is set to its current location plus offset bytes.
   • Useful for skipping data or moving backwards relatively.
   • Example: lseek(fd, 10, SEEK_CUR) skips the next 10 bytes.

3. SEEK_END:

   • The file offset is set to the size of the file plus offset bytes.
   • Used to append data or find file size.
   • Example: lseek(fd, 0, SEEK_END) moves to the end of the file.

System Calls vs. Library Functions

Importance of close()

The close(int fd) system call dissociates the file descriptor from the corresponding file, releasing the resources allocated by the kernel for that connection.

Consequences of Failing to Close

1. Resource Leakage: Operating systems have a limit on the maximum number of file descriptors a single process can have open simultaneously (often 1024 by default). If a program repeatedly opens files without closing them, it will exhaust the File Descriptor Table.
2. "Too many open files" Error: Once the limit is reached, subsequent open() calls will fail, causing the program to crash or malfunction.
3. Data Integrity (Buffering): While less critical for unbuffered write, in some contexts, closing a file ensures that any pending kernel buffers associated with the file logic are properly flushed or committed.
4. Locks: If the file was locked by the process, failing to close it might keep the lock active, preventing other processes from accessing the file.

To determine the size of a file using lseek, we can move the file pointer to the end of the file. The return value of lseek is the new offset from the beginning of the file, which, when positioned at the end, equals the total file size in bytes.

Logic:

1. Open the file.
2. Use lseek with SEEK_END and offset 0.
3. The return value is the size.
4. Optionally, reset position to SEEK_SET if further reading is required.

Code Snippet:
c

include <fcntl.h>

include <unistd.h>

include <stdio.h>

int main() {
int fd = open("example.txt", O_RDONLY);
if (fd == -1) {
perror("Open error");
return 1;
}

// Move to end to get size
off_t fileSize = lseek(fd, 0, SEEK_END);

if (fileSize == -1)
    perror("Lseek error");
else
    printf("File size: %ld bytes\n", (long)fileSize);

close(fd);
return 0;

}

File Holes (Sparse Files)

A file hole occurs when the file offset is repositioned past the current end of the file using lseek(), and data is subsequently written at this new location. The gap between the old end-of-file and the new write position is called a "hole."

• Storage: The file system does not allocate physical disk blocks for the hole (the bytes containing zeros). This saves disk space.
• Reading: If a process reads from the hole, the file system returns a buffer filled with null bytes ($0$).

Creation Method

1. Write some data to the file.
2. lseek() past the end of the file (e.g., advance offset by 1000 bytes).
3. Write data again at the new offset.

Example Sequence:
c
write(fd, "Start", 5); // Writes at offset 0-5
lseek(fd, 100, SEEK_END); // Moves pointer to 105 (hole from 5 to 105)
write(fd, "End", 3); // Writes at 105-108

Physically, the file consumes space only for "Start" and "End", but the logical size is 108 bytes.

The mode Argument

The mode argument in the open(pathname, flags, mode) system call specifies the file access permissions (permissions for User, Group, and Others).

Mandatory Usage:
It is mandatory to supply the mode argument when the O_CREAT flag is used in the flags argument. If O_CREAT is used without mode, the permissions of the newly created file will be undefined (garbage values), potentially creating security vulnerabilities.

Common Modes:
It is often specified using octal numbers or symbolic constants:

• 0777 (rwxrwxrwx)
• 0644 (rw-r--r--)
• Constants: S_IRUSR (User Read), S_IWUSR (User Write), S_IXUSR (User Execute), etc.

Note: The actual permissions set are modified by the process's umask (mode & ~umask).

c

include <fcntl.h>

include <unistd.h>

include <stdio.h>

include <stdlib.h>

int main(int argc, char *argv[]) {
if (argc != 2) {
printf("Usage: %s <filename>\n", argv[0]);
exit(1);
}

int fd = open(argv[1], O_RDONLY);
if (fd == -1) {
    perror("Error opening file");
    exit(1);
}

// Move pointer to 10 bytes before the end
// Note: We should check if file size >= 10
off_t fileSize = lseek(fd, 0, SEEK_END);

if (fileSize < 10) {
    lseek(fd, 0, SEEK_SET); // If file is small, read from start
} else {
    lseek(fd, -10, SEEK_END); // Move 10 bytes back from end
}

char buffer[11];
ssize_t bytesRead = read(fd, buffer, 10);

if (bytesRead > 0) {
    buffer[bytesRead] = '\0'; // Null terminate for printing
    // Write to STDOUT (File Descriptor 1)
    write(1, buffer, bytesRead);
    write(1, "\n", 1);
}

close(fd);
return 0;

}

Purpose of O_APPEND

The O_APPEND flag ensures that all write operations on the file descriptor are performed at the end of the file, regardless of the current file pointer position.

• Before each write(), the file offset is positioned at the end of the file.

Atomicity and Race Conditions

Without O_APPEND, a process trying to append to a log file shared by multiple processes might do this:

1. lseek(fd, 0, SEEK_END)
2. write(fd, buffer, len)

If a context switch occurs between step 1 and 2, another process might write data. When the first process resumes, it overwrites the data written by the second process.

With O_APPEND, the kernel guarantees that the positioning at the end and the writing happen as a single atomic operation. This prevents data overwrites in concurrent logging scenarios.

write() System Call

Syntax: ssize_t write(int fd, const void *buf, size_t count);

Parameters:

1. fd: The file descriptor where data is to be written.
2. buf: Pointer to the buffer containing the data to be written.
3. count: The number of bytes to write from the buffer.

Partial Writes

The function returns the number of bytes actually written. Usually, this equals count. However, a return value less than count (but ) can occur due to:

• Disk Full: The underlying storage device is out of space.
• Resource Limits: The file size limit for the process (RLIMIT_FSIZE) is reached.
• Interruption: The call was interrupted by a signal handler after writing some data (mostly relevant for slow devices like terminals or sockets, less common for disk files).

If this happens, the programmer usually needs to write a loop to write the remaining bytes.

O_TRUNC Flag

The O_TRUNC (Truncate) flag is used in the open() system call, typically in conjunction with O_WRONLY or O_RDWR.

Effect:
If the file:

1. Exists,
2. Is a regular file, and
3. The open mode allows writing,

...then the file is successfully opened, and its length is truncated to 0.

Implication:

• All previous data in the file is deleted immediately.
• The file pointer is set to the beginning of the file.
• This is commonly used when you want to overwrite a file completely (e.g., "Save" functionality in a text editor).

To read from the keyboard without standard I/O library functions, we can use the read() system call on the Standard Input file descriptor, which is defined as 0 (or STDIN_FILENO from <unistd.h>).

Example:
c

include <unistd.h>

include <stdio.h>

int main() {
char buffer[100];
ssize_t bytesRead;

// Write prompt to screen (fd 1)
write(1, "Enter text: ", 12);

// Read from Keyboard (fd 0)
// This will block until user presses Enter
bytesRead = read(0, buffer, sizeof(buffer));

if (bytesRead > 0) {
    write(1, "You typed: ", 11);
    write(1, buffer, bytesRead);
}

return 0;

}

To strictly comply with POSIX standards for these system calls, the following headers are required:

1. <fcntl.h>: Required for the open() system call and the flag definitions (e.g., O_RDONLY, O_CREAT, O_TRUNC).
2. <unistd.h>: Required for read(), write(), close(), lseek(), and standard file descriptor constants (STDIN_FILENO, etc.).
3. <stdlib.h>: Required for the exit() function.
4. <sys/types.h> and <sys/stat.h>: Often included for data types like mode_t or off_t and file status flags, though modern compilers/standards sometimes include these implicitly via other headers.

To check if a file exists without modifying it or changing its timestamps significantly (other than access time), we can attempt to open it for reading. However, permissions might block reading.

A robust way using open() is to use the O_RDONLY flag. A more specific way (if just checking existence and permissions are irrelevant for the check) relies on how error codes are handled, but strictly using open:

Logic:

1. Call open("filename", O_RDONLY).
2. Check Return Value:
   • If fd >= 0: The file exists and we have read permissions. Close the fd immediately.
   • If fd == -1 and errno == ENOENT: The file does not exist.
   • If fd == -1 and errno == EACCES: The file exists, but we don't have permission to read it.

Alternatively, open("filename", O_CREAT | O_EXCL, mode) returns an error if the file already exists, which is another way to check existence (usually used to ensure creating a new file).

Relationship between FD Table and System File Table

1. User File Descriptor Table:

   • Each process has its own table.
   • The File Descriptor (int) returned by open acts as an index into this table.
   • The entry at this index points to an entry in the System-wide File Table.

2. System-wide File Table (Open File Table):

   • Maintained by the OS kernel.
   • Contains entries for every open file in the system.
   • Stores the Current File Offset (read/write pointer), File Status Flags (read/write mode), and a pointer to the V-node (Inode) table.

Significance:

• Two different processes can open the same file. They will have distinct file descriptors and distinct entries in the System File Table (so they have independent file offsets), but both will point to the same V-node (physical file location).

lseek() on Pipes/Sockets

Behavior:
If lseek() is called on a file descriptor referring to a pipe, FIFO (named pipe), or socket, the system call fails.

Return Value:
It returns (off_t) -1.

Error Code:
The global variable errno is set to ESPIPE (Illegal seek).

Reasoning:

• Pipes, FIFOs, and Sockets are sequential data streams. Data is read in the order it arrives and is consumed (removed) upon reading.
• There is no concept of a persistent "file position" that can be rewound or advanced arbitrarily, as the data is not stored permanently on a disk addressable by offset.

This technique involves replacing file descriptor 1 (stdout) with a file descriptor for log.txt.

c

include <fcntl.h>

include <unistd.h>

include <stdio.h>

int main() {
// 1. Open the file to direct output to
int logFd = open("log.txt", O_WRONLY | O_CREAT | O_TRUNC, 0644);
if (logFd == -1) {
perror("Failed to open log file");
return 1;
}

// 2. Redirect stdout (fd 1) to logFd
// close(1) followed by dup(logFd) ensures the new fd is 1.
// Alternatively, dup2(logFd, 1) does this atomically.

if (dup2(logFd, STDOUT_FILENO) == -1) {
    perror("Failed to redirect stdout");
    return 1;
}

// 3. Close the original file descriptor (no longer needed)
close(logFd);

// 4. Test redirection
printf("This text goes to log.txt instead of the screen.\n");

// flush is important for buffered IO like printf before exit
fflush(stdout); 

return 0;

}