System Utilities
****************

Helper functions for working with the underlying system. These are
mostly os dependent, only working on linux, osx, and bsd. In almost
all cases they’re best-effort, providing **None** if the lookup fails.

Changed in version 1.3.0: Dropped the get_* prefix from several
function names. The old names still work, but are deprecated aliases.

Changed in version 1.5.0: Added the **SYSTEM_CALL_TIME** global, which
tracks total time spent making system commands.

**Module Overview:**

   is_windows - checks if we're running on windows
   is_mac - checks if we're running on a mac
   is_gentoo - checks if we're running on gentoo
   is_slackware - checks if we're running on slackware
   is_bsd - checks if we're running on the bsd family of operating systems

   is_available - determines if a command is available on this system
   is_running - determines if a given process is running
   size_of - provides the memory usage of an object
   call - runs the given system command and provides back the results

   name_by_pid - gets the name for a process by the given pid
   pid_by_name - gets the pid for a process by the given name
   pid_by_port - gets the pid for a process listening to a given port
   pid_by_open_file - gets the pid for the process with an open file
   pids_by_user - provides processes owned by a user
   cwd - provides the current working directory for a given process
   user - provides the user a process is running under
   start_time - provides the unix timestamp when the process started
   tail - provides lines from the end of a file
   bsd_jail_id - provides the BSD jail id a given process is running within
   bsd_jail_path - provides the path of the given BSD jail

   is_tarfile - checks if the given path is a tarball
   expand_path - expands relative paths and ~ entries
   files_with_suffix - provides files with the given suffix

   get_process_name - provides our process' name
   set_process_name - changes our process' name

stem.util.system.Status(enum)

   State of a subprocess.

   New in version 1.6.0.

   +----------------------+---------------------------+
   | Status               | Description               |
   +======================+===========================+
   | PENDING              | not yet started           |
   +----------------------+---------------------------+
   | RUNNING              | currently being performed |
   +----------------------+---------------------------+
   | DONE                 | completed successfully    |
   +----------------------+---------------------------+
   | FAILED               | failed with an exception  |
   +----------------------+---------------------------+

stem.util.system.argc_t

   alias of "stem.util.system.LP_c_char_p"

exception stem.util.system.CallError(msg, command, exit_status, runtime, stdout, stderr)

   Bases: "OSError"

   Error response when making a system call. This is an **OSError**
   subclass with additional information about the process. Depending
   on the nature of the error not all of these attributes will be
   available.

   Variables:
      * **msg** (*str*) – exception string

      * **command** (*str*) – command that was ran

      * **exit_status** (*int*) – exit code of the process

      * **runtime** (*float*) – time the command took to run

      * **stdout** (*str*) – stdout of the process

      * **stderr** (*str*) – stderr of the process

exception stem.util.system.CallTimeoutError(msg, command, exit_status, runtime, stdout, stderr, timeout)

   Bases: "stem.util.system.CallError"

   Error response when making a system call that has timed out.

   New in version 1.6.0.

   Variables:
      **timeout** (*float*) – time we waited

class stem.util.system.DaemonTask(runner, args=None, priority=15, start=False)

   Bases: "object"

   Invokes the given function in a subprocess, returning the value.

   New in version 1.6.0.

   Variables:
      * **runner** (*function*) – function to be invoked by the
        subprocess

      * **args** (*tuple*) – arguments to provide to the subprocess

      * **priority** (*int*) – subprocess nice priority

      * **status** (*stem.util.system.State*) – state of the
        subprocess

      * **runtime** (*float*) – seconds subprocess took to complete

      * **result** (*object*) – return value of subprocess if
        successful

      * **error** (*exception*) – exception raised by subprocess if
        it failed

   run()

      Invokes the task if it hasn’t already been started. If it has
      this is a no-op.

   join()

      Provides the result of the daemon task. If still running this
      blocks until the task is completed.

      Returns:
         response of the function we ran

      Raises:
         exception raised by the function if it failed with one

stem.util.system.is_windows()

   Checks if we are running on Windows.

   Returns:
      **bool** to indicate if we’re on Windows

stem.util.system.is_mac()

   Checks if we are running on Mac OSX.

   Returns:
      **bool** to indicate if we’re on a Mac

stem.util.system.is_gentoo()

   Checks if we’re running on Gentoo.

   Returns:
      **bool** to indicate if we’re on Gentoo

stem.util.system.is_slackware()

   Checks if we are running on a Slackware system.

   Returns:
      **bool** to indicate if we’re on a Slackware system

stem.util.system.is_bsd()

   Checks if we are within the BSD family of operating systems. This
   currently recognizes Macs, FreeBSD, and OpenBSD but may be expanded
   later.

   Returns:
      **bool** to indicate if we’re on a BSD OS

stem.util.system.is_available(command, cached=True)

   Checks the current PATH to see if a command is available or not. If
   more than one command is present (for instance “ls -a | grep foo”)
   then this just checks the first.

   Note that shell (like cd and ulimit) aren’t in the PATH so this
   lookup will try to assume that it’s available. This only happends
   for recognized shell commands (those in SHELL_COMMANDS).

   Parameters:
      * **command** (*str*) – command to search for

      * **cached** (*bool*) – makes use of available cached results
        if **True**

   Returns:
      **True** if an executable we can use by that name exists in the
      PATH, **False** otherwise

stem.util.system.is_running(command)

   Checks for if a process with a given name or pid is running.

   Changed in version 1.6.0: Added support for list and pid arguments.

   Parameters:
      **command** (*str**,**list**,**int*) – process name if a str,
      multiple process names if a list, or pid if an int to be checked

   Returns:
      **True** if the process is running, **False** if it’s not among
      ps results, and **None** if ps can’t be queried

stem.util.system.size_of(obj, exclude=None)

   Provides the approximate memory usage of an object. This can
   recurse tuples, lists, deques, dicts, and sets. To teach this
   function to inspect additional object types expand SIZE_RECURSES…

      stem.util.system.SIZE_RECURSES[SomeClass] = SomeClass.get_elements

   New in version 1.6.0.

   Parameters:
      * **obj** (*object*) – object to provide the size of

      * **exclude** (*set*) – object ids to exclude from size
        estimation

   Returns:
      **int** with the size of the object in bytes

   Raises:
      **NotImplementedError** if using PyPy

stem.util.system.name_by_pid(pid)

   Attempts to determine the name a given process is running under
   (not including arguments). This uses…

      1. Information from /proc
      2. ps -p <pid> -o command

   Parameters:
      **pid** (*int*) – process id of the process to be queried

   Returns:
      **str** with the process name, **None** if it can’t be
      determined

stem.util.system.pid_by_name(process_name, multiple=False)

   Attempts to determine the process id for a running process, using…

      1. pgrep -x <name>
      2. pidof <name>
      3. ps -o pid -C <name> (linux)
         ps axc | egrep " <name>$" (bsd)
      4. lsof -tc <name>
      5. tasklist | str <name>.exe

   Parameters:
      * **process_name** (*str*) – process name for which to fetch
        the pid

      * **multiple** (*bool*) – provides a list of all pids if
        **True**, otherwise results with multiple processes are
        discarded

   Returns:
      Response depends upon the ‘multiple’ argument as follows…

      * if **False** then this provides an **int** with the process
        id or **None** if it can’t be determined

      * if **True** then this provides a **list** of all **int**
        process ids, and an empty list if it can’t be determined

stem.util.system.pid_by_port(port)

   Attempts to determine the process id for a process with the given
   port, using…

      1. netstat -npltu | grep 127.0.0.1:<port>
      2. sockstat -4l -P tcp -p <port>
      3. lsof -wnP -iTCP -sTCP:LISTEN | grep ":<port>"

   Most queries limit results to listening TCP connections. This
   function likely won’t work on Mac OSX.

   Parameters:
      **port** (*int*) – port where the process we’re looking for is
      listening

   Returns:
      **int** with the process id, **None** if it can’t be determined

stem.util.system.pid_by_open_file(path)

   Attempts to determine the process id for a process with the given
   open file, using…

      lsof -w <path>

   Parameters:
      **path** (*str*) – location of the socket file to query against

   Returns:
      **int** with the process id, **None** if it can’t be determined

stem.util.system.pids_by_user(user)

   Provides processes owned by a given user.

   New in version 1.5.0.

   Parameters:
      **user** (*str*) – user to look up processes for

   Returns:
      **list** with the process ids, **None** if it can’t be
      determined

stem.util.system.cwd(pid)

   Provides the working directory of the given process.

   Parameters:
      **pid** (*int*) – process id of the process to be queried

   Returns:
      **str** with the absolute path for the process’ present working
      directory, **None** if it can’t be determined

stem.util.system.user(pid)

   Provides the user a process is running under.

   Parameters:
      **pid** (*int*) – process id of the process to be queried

   Returns:
      **str** with the username a process is running under, **None**
      if it can’t be determined

stem.util.system.start_time(pid)

   Provides the unix timestamp when the given process started.

   Parameters:
      **pid** (*int*) – process id of the process to be queried

   Returns:
      **float** for the unix timestamp when the process began,
      **None** if it can’t be determined

stem.util.system.tail(target, lines=None)

   Provides lines of a file starting with the end. For instance, ‘tail
   -n 50 /tmp/my_log’ could be done with…

      reversed(list(tail('/tmp/my_log', 50)))

   Parameters:
      * **target** (*str**,**file*) – path or file object to read
        from

      * **lines** (*int*) – number of lines to read

   Returns:
      **generator** that reads lines, starting with the end

   Raises:
      **IOError** if unable to read the file

stem.util.system.bsd_jail_id(pid)

   Gets the jail id for a process. These seem to only exist for
   FreeBSD (this style for jails does not exist on Linux, OSX, or
   OpenBSD).

   Parameters:
      **pid** (*int*) – process id of the jail id to be queried

   Returns:
      **int** for the jail id, zero if this can’t be determined

stem.util.system.bsd_jail_path(jid)

   Provides the path of the given FreeBSD jail.

   Parameters:
      **jid** (*int*) – jail id to be queried

   Returns:
      **str** of the path prefix, **None** if this can’t be determined

stem.util.system.is_tarfile(path)

   Returns if the path belongs to a tarfile or not.

   New in version 1.2.0.

   Parameters:
      **path** (*str*) – path to be checked

   Returns:
      **True** if the path belongs to a tarball, **False** otherwise

stem.util.system.expand_path(path, cwd=None)

   Provides an absolute path, expanding tildes with the user’s home
   and appending a current working directory if the path was relative.

   Parameters:
      * **path** (*str*) – path to be expanded

      * **cwd** (*str*) – current working directory to expand
        relative paths with, our process’ if this is **None**

   Returns:
      **str** of the path expanded to be an absolute path, never with
      an ending slash

stem.util.system.files_with_suffix(base_path, suffix)

   Iterates over files in a given directory, providing filenames with
   a certain suffix.

   New in version 1.2.0.

   Parameters:
      * **base_path** (*str*) – directory to be iterated over

      * **suffix** (*str*) – filename suffix to look for

   Returns:
      iterator that yields the absolute path for files with the given
      suffix

stem.util.system.call(command, default = UNDEFINED, ignore_exit_status = False)

   Issues a command in a subprocess, blocking until completion and
   returning the results. This is not actually ran in a shell so pipes
   and other shell syntax are not permitted.

   Changed in version 1.5.0: Providing additional information upon
   failure by raising a CallError. This is a subclass of OSError,
   providing backward compatibility.

   Changed in version 1.5.0: Added env argument.

   Changed in version 1.6.0: Added timeout and cwd arguments.

   Parameters:
      * **command** (*str**,**list*) – command to be issued

      * **default** (*object*) – response if the query fails

      * **ignore_exit_status** (*bool*) – reports failure if our
        command’s exit status was non-zero

      * **timeout** (*float*) – maximum seconds to wait, blocks
        indefinitely if **None**

      * **env** (*dict*) – environment variables

   Returns:
      **list** with the lines of output from the command

   Raises:
      * **CallError** if this fails and no default was provided

      * **CallTimeoutError** if the timeout is reached without a
        default

stem.util.system.get_process_name()

   Provides the present name of our process.

   Returns:
      **str** with the present name of our process

stem.util.system.set_process_name(process_name)

   Renames our current process from “python <args>” to a custom name.
   This is best-effort, not necessarily working on all platforms.

   Parameters:
      **process_name** (*str*) – new name for our process

   Raises:
      **IOError** if the process cannot be renamed

stem.util.system.get_name_by_pid(pid)

   Attempts to determine the name a given process is running under
   (not including arguments). This uses…

      1. Information from /proc
      2. ps -p <pid> -o command

   Parameters:
      **pid** (*int*) – process id of the process to be queried

   Returns:
      **str** with the process name, **None** if it can’t be
      determined

stem.util.system.get_pid_by_name(process_name, multiple=False)

   Attempts to determine the process id for a running process, using…

      1. pgrep -x <name>
      2. pidof <name>
      3. ps -o pid -C <name> (linux)
         ps axc | egrep " <name>$" (bsd)
      4. lsof -tc <name>
      5. tasklist | str <name>.exe

   Parameters:
      * **process_name** (*str*) – process name for which to fetch
        the pid

      * **multiple** (*bool*) – provides a list of all pids if
        **True**, otherwise results with multiple processes are
        discarded

   Returns:
      Response depends upon the ‘multiple’ argument as follows…

      * if **False** then this provides an **int** with the process
        id or **None** if it can’t be determined

      * if **True** then this provides a **list** of all **int**
        process ids, and an empty list if it can’t be determined

stem.util.system.get_pid_by_port(port)

   Attempts to determine the process id for a process with the given
   port, using…

      1. netstat -npltu | grep 127.0.0.1:<port>
      2. sockstat -4l -P tcp -p <port>
      3. lsof -wnP -iTCP -sTCP:LISTEN | grep ":<port>"

   Most queries limit results to listening TCP connections. This
   function likely won’t work on Mac OSX.

   Parameters:
      **port** (*int*) – port where the process we’re looking for is
      listening

   Returns:
      **int** with the process id, **None** if it can’t be determined

stem.util.system.get_pid_by_open_file(path)

   Attempts to determine the process id for a process with the given
   open file, using…

      lsof -w <path>

   Parameters:
      **path** (*str*) – location of the socket file to query against

   Returns:
      **int** with the process id, **None** if it can’t be determined

stem.util.system.get_cwd(pid)

   Provides the working directory of the given process.

   Parameters:
      **pid** (*int*) – process id of the process to be queried

   Returns:
      **str** with the absolute path for the process’ present working
      directory, **None** if it can’t be determined

stem.util.system.get_user(pid)

   Provides the user a process is running under.

   Parameters:
      **pid** (*int*) – process id of the process to be queried

   Returns:
      **str** with the username a process is running under, **None**
      if it can’t be determined

stem.util.system.get_start_time(pid)

   Provides the unix timestamp when the given process started.

   Parameters:
      **pid** (*int*) – process id of the process to be queried

   Returns:
      **float** for the unix timestamp when the process began,
      **None** if it can’t be determined

stem.util.system.get_bsd_jail_id(pid)

   Gets the jail id for a process. These seem to only exist for
   FreeBSD (this style for jails does not exist on Linux, OSX, or
   OpenBSD).

   Parameters:
      **pid** (*int*) – process id of the jail id to be queried

   Returns:
      **int** for the jail id, zero if this can’t be determined

stem.util.system.get_bsd_jail_path(jid)

   Provides the path of the given FreeBSD jail.

   Parameters:
      **jid** (*int*) – jail id to be queried

   Returns:
      **str** of the path prefix, **None** if this can’t be determined
