I need to strace the output of any command through python. But I only need to translate the following amd linux command to mac m1/arm commands (so python is likely irrelevant):

strace -e trace=execve -v -s 100000000 -xx -ttt -ff -o output.txt sh -c 'echo hi' 

How do I do that?

This fails for me:

 ❯ sudo dtruss -t execve -f sh -c 'echo hi' dtrace: system integrity protection is on, some features will not be available dtrace: failed to execute sh: Operation not permitted 

Note:

• I have complete control of the input so I can do sudo and related commands in my mac (it's mainly to debug my code, so it works on pycharm)

I can't seem to install strace from brew:

 ❯ brew install strace Running `brew update --auto-update`... strace: Linux is required for this software. [email protected]: Linux is required for this software. Error: strace: Unsatisfied requirements failed this build. 

❯ sudo execsnoop sh -c 'echo hi' dtrace: system integrity protection is on, some features will not be available dtrace: invalid probe specifier /* * Command line arguments */ inline int OPT_dump = 0; inline int OPT_cmd = 0; inline int OPT_time = 0; inline int OPT_timestr = 0; inline int OPT_zone = 0; inline int OPT_safe = 0; inline int OPT_proj = 0; inline int FILTER = 0; inline string COMMAND = "."; #pragma D option quiet #pragma D option switchrate=10hz /* * Print header */ dtrace:::BEGIN { /* print optional headers */ OPT_time ? printf("%-14s ", "TIME") : 1; OPT_timestr ? printf("%-20s ", "STRTIME") : 1; OPT_zone ? printf("%-10s ", "ZONE") : 1; OPT_proj ? printf("%5s ", "PROJ") : 1; /* print main headers */ /* APPLE: Removed "ZONE" header, it has no meaning in darwin */ OPT_dump ? printf("%s %s %s %s %s %s %s\n", "TIME", "PROJ", "UID", "PID", "PPID", "COMM", "ARGS") : printf("%5s %6s %6s %s\n", "UID", "PID", "PPID", "ARGS"); } /* * Print exec event */ /* SOLARIS: syscall::exec:return, syscall::exece:return */ proc:::exec-success /(FILTER == 0) || (OPT_cmd == 1 && COMMAND == strstr(COMMAND, execname)) || (OPT_cmd == 1 && execname == strstr(execname, COMMAND))/ { /* print optional fields */ OPT_time ? printf("%-14d ", timestamp/1000) : 1; OPT_timestr ? printf("%-20Y ", walltimestamp) : 1; OPT_zone ? printf("%-10s ", zonename) : 1; OPT_proj ? printf("%5d ", curpsinfo->pr_projid) : 1; /* print main data */ /* APPLE: Removed the zonename output, it has no meaning in darwin */ OPT_dump ? printf("%d %d %d %d %d %s ", timestamp/1000, curpsinfo->pr_projid, uid, pid, ppid, execname) : printf("%5d %6d %6d ", uid, pid, ppid); OPT_safe ? printf("%S\n", curpsinfo->pr_psargs) : printf("%s\n", curpsinfo->pr_psargs); } : probe description proc:::exec-success does not match any probes. System Integrity Protection is on 

I inherited this code and in it calls strace from within python. In particular it calls:

def strace_build(executable: str, regex: str, workdir: Optional[str], command: List[str], strace_logdir=None) -> List[str]: ''' trace calls of executable during access to files that match regex in workdir while executing the command and returns the list of pycoq_context file names In the simplest case strace runs the specified command until it exits. It intercepts and records the system calls which are called by a process and the signals which are received by a process. The name of each system call, its arguments and its return value are printed on standard error or to the file specified with the -o option. ''' print('---- Calling strace_build ----') def _strace_build(executable, regex, workdir, command, logdir): logfname = os.path.join(logdir, 'strace.log') logging.info(f"pycoq: tracing {executable} accesing {regex} while " f"executing {command} from {workdir} with " f"curdir {os.getcwd()}") print(f"pycoq: tracing {executable} accesing {regex} while " f"executing {command} from {workdir} with " f"curdir {os.getcwd()}") with subprocess.Popen(['strace', '-e', 'trace=execve', '-v', '-ff', '-s', '100000000', '-xx', '-ttt', '-o', logfname] + command, cwd=workdir, text=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) as proc: for line in iter(proc.stdout.readline, ''): logging.debug(f"strace stdout: {line}") print(f"strace stdout: {line=}") logging.info(f"strace stderr: {proc.stderr.read()}" "waiting strace to finish...") proc.wait() logging.info('strace finished') res: list[str] = parse_strace_logdir(logdir, executable, regex) print('---- Done with strace_build ----') return res if strace_logdir is None: with tempfile.TemporaryDirectory() as _logdir: return _strace_build(executable, regex, workdir, command, _logdir) else: os.makedirs(strace_logdir, exist_ok=True) strace_logdir_cur = tempfile.mkdtemp(dir=strace_logdir) return _strace_build(executable, regex, workdir, command, strace_logdir_cur) 

but because it calls strace it only works on linux. I want it to work on my mac -- ideally if possible in the most pythonic way possible. I believe what it does is strace a terminal command that is called from within python.

What would be an equivalent way to call this command on mac using the same flags so that it works (ideally exactly) the same?

Not sure if this matters but I am using an m1 mac.

some output when the above function is used:

--done with make attempt-- ---- Calling strace_build ---- pycoq: tracing /home/bot/.opam/ocaml-variants.4.07.1+flambda_coq-serapi.8.11.0+0.11.1/bin/coqc accesing .*\.v$ while executing ['opam', 'reinstall', '--yes', '--switch', 'ocaml-variants.4.07.1+flambda_coq-serapi.8.11.0+0.11.1', '--keep-build-dir', 'debug_proj'] from None with curdir /home/bot strace stdout: line='\n' strace stdout: line='<><> Synchronising pinned packages ><><><><><><><><><><><><><><><><><><><><><><>\n' strace stdout: line='[debug_proj.~dev] no changes from file:///home/bot/iit-term-synthesis/coq_projects/debug_proj\n' strace stdout: line='\n' strace stdout: line='debug_proj is not installed. Install it? [Y/n] y\n' strace stdout: line='Sorry, no solution found: there seems to be a problem with your request.\n' strace stdout: line='\n' strace stdout: line='No solution found, exiting\n' ---- Done with strace_build ---- ... ---- Calling strace_build ---- pycoq: tracing /home/bot/.opam/ocaml-variants.4.07.1+flambda_coq-serapi.8.11.0+0.11.1/bin/coqc accesing .*\.v$ while executing ['make', '-C', '/home/bot/iit-term-synthesis/coq_projects/debug_proj'] from None with curdir /home/bot strace stdout: line="make: Entering directory '/home/bot/iit-term-synthesis/coq_projects/debug_proj'\n" strace stdout: line='coq_makefile -f _CoqProject -o CoqMakefile\n' strace stdout: line='make --no-print-directory -f CoqMakefile \n' strace stdout: line='COQDEP VFILES\n' strace stdout: line='COQC debug_0_plus_n_eq_n.v\n' strace stdout: line='COQC debug1_n_plus_1_greater_than_n.v\n' strace stdout: line='COQC debug2_n_plus_0_eq_n.v\n' strace stdout: line="make: Leaving directory '/home/bot/iit-term-synthesis/coq_projects/debug_proj'\n" ---- Done with strace_build ---- 

def strace_build_mac_m1(executable: str, regex: str, workdir: Optional[str], command: List[str], strace_logdir=None) -> List[str]: ''' trace calls of executable during access to files that match regex in workdir while executing the command and returns the list of pycoq_context file names In the simplest case strace runs the specified command until it exits. It intercepts and records the system calls which are called by a process and the signals which are received by a process. The name of each system call, its arguments and its return value are printed on standard error or to the file specified with the -o option. plan: - get the command we are running - pip push my pycoq with no name changes so code doesn't break - pull the rest of the repos needed, I don't think anything else since lf is here - harcode test - actually, look at commands...we need to provide for reproducibility a way to install opam and all this stuff without docker but in the mac since we are trying to do a mac install. Argh... COMMANDS: pycoq: tracing /home/bot/.opam/ocaml-variants.4.07.1+flambda_coq-serapi.8.11.0+0.11.1/bin/coqc accesing .*\.v$ while executing ['opam', 'reinstall', '--yes', '--switch', 'ocaml-variants.4.07.1+flambda_coq-serapi.8.11.0+0.11.1', '--keep-build-dir', 'lf'] from None with curdir /home/bot executable='/home/bot/.opam/ocaml-variants.4.07.1+flambda_coq-serapi.8.11.0+0.11.1/bin/coqc' regex='.*\\.v$' workdir=None command=['opam', 'reinstall', '--yes', '--switch', 'ocaml-variants.4.07.1+flambda_coq-serapi.8.11.0+0.11.1', '--keep-build-dir', 'lf'] curdir: os.getcwd()='/home/bot' ''' print('---- Calling strace_build_mac_m1 ----') def _strace_build(executable, regex, workdir, command, logdir): logfname = os.path.join(logdir, 'strace.log') logging.info(f"pycoq: tracing {executable} accesing {regex} while " f"executing {command} from {workdir} with " f"curdir {os.getcwd()}") print(f"pycoq: tracing {executable} accesing {regex} while " f"executing {command} from {workdir} with " f"curdir {os.getcwd()}") print(f'{executable=}') print(f'{regex=}') print(f'{workdir=}') print(f'{command=}') print(f'curdir: {os.getcwd()=}') with subprocess.Popen(['dtruss', '-e', 'trace=execve', '-v', '-ff', '-s', '100000000', '-xx', '-ttt', '-o', logfname] + command, cwd=workdir, text=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) as proc: for line in iter(proc.stdout.readline, ''): logging.debug(f"strace stdout: {line}") print(f"strace stdout: {line=}") logging.info(f"strace stderr: {proc.stderr.read()}" "waiting strace to finish...") proc.wait() logging.info('strace finished') res: list[str] = parse_strace_logdir(logdir, executable, regex) print('---- Done with strace_build_mac_m1 ----') return res if strace_logdir is None: with tempfile.TemporaryDirectory() as _logdir: return _strace_build(executable, regex, workdir, command, _logdir) else: os.makedirs(strace_logdir, exist_ok=True) strace_logdir_cur = tempfile.mkdtemp(dir=strace_logdir) return _strace_build(executable, regex, workdir, command, strace_logdir_cur) # - def code_for_mac_m1(): coq_package = 'lf' coq_package_pin = '~/pycoq/pycoq/test/lf' coq_package_pin = os.path.expanduser(coq_package_pin) print(f'coq_package: {coq_package=}') print(f'coq_package_pin: {coq_package_pin=}') ### pycoq: tracing /home/bot/.opam/ocaml-variants.4.07.1+flambda_coq-serapi.8.11.0+0.11.1/bin/coqc accesing .*\.v$ while executing ['opam', 'reinstall', '--yes', '--switch', 'ocaml-variants.4.07.1+flambda_coq-serapi.8.11.0+0.11.1', '--keep-build-dir', 'lf'] from None with curdir /home/bot # executable='/home/bot/.opam/ocaml-variants.4.07.1+flambda_coq-serapi.8.11.0+0.11.1/bin/coqc' # regex='.*\\.v$' # workdir=None # command=['opam', 'reinstall', '--yes', '--switch', 'ocaml-variants.4.07.1+flambda_coq-serapi.8.11.0+0.11.1', '--keep-build-dir', 'lf'] # curdir: os.getcwd()='/home/bot' # - get the filename in split # path2filenames: list[str] = pycoq.opam.opam_strace_build(coq_proj, coq_proj_pin) path2filenames_raw: list[str] = strace_build_mac_m1() path2filenames_raw.sort() print(f'\n====----> Populate coq pkg/proj data with files: {path2filenames_raw=}') if __name__ == '__main__': code_for_mac_m1() print('Done!\n\a') 

why doesn't this meet the SO guidelines? I am not asking for a software request clearly. From the discussion it's much harder to run in python + having the mac setup as needed + the tool with the equivalent flags to strace for mac.

google for replacement (analogue) of functionality of strace for Mac OS and modify the parser that parses output of strace for that “mac-strace” The strace functionality is used to inspect the Coq building system to record all options and arguments and environment for coqc in which each individual .v file has be processed by coqc during the build process. The Coq build system is complicated, and i didn’t parse but resolved to just observing by strace of what the actual Coq does and strace simply records all options and arguments (like -R etc) of what the actual Coq does so that pycoq could call coqc with exactly the same options and arguments

Answer by philipe didn't work:

(meta_learning) brandomiranda~ ❯ cat dtruss.sh #!/bin/bash file=$(type -P $1); shift c=/tmp/$(basename "$file") cp "$file" "$c" codesign --remove-signature "$c" sudo dtruss "$c" "$@" (meta_learning) brandomiranda~ ❯ chmod +x dtruss.sh (meta_learning) brandomiranda~ ❯ ./dtruss.sh echo hi dtrace: system integrity protection is on, some features will not be available dtrace: failed to execute /tmp/echo: Could not create symbolicator for task 

To fix sudo to not prompt for password see:

# root and users in group wheel can run anything on any machine as any user root ALL = (ALL) ALL %admin ALL = (ALL) NOPASSWD: ALL #%admin ALL = (ALL) ALL 

Related:

6 Answers

Most of the answers and workarounds in the question are attempting to work inside the Linux security model. For a number of year Apple has steadily increased their security beyond their Mach/BSD roots. They have added Hardware security that hardware root of trust from power on to the full macOS load. The Secure Enclave also processes face and fingerprint data. FileVault can encrypt your entire disk. System Integrity Protection provides a rootless system where root has little more permissions than the nobody user. For example, on a Ventura 13.2 system root can not see every file or directory (sudo find / -print|wc produces many "Operation not permitted" errors).

In short, the only viable method to trace execution like this has a number of hefty requirements:

• You must use Apple's EndpointSecurity framework.
• Your code must be compiled in Xcode as an Application Bundle (Program.app installed in /Applications).
• You will need Apple to grant your developer account the 'com.apple.developer.endpoint-security.client' entitlement.
• You will need to assign a Provisioning Profile to codesign with the granted entitlement.
• If you have not been granted the entitlement, you must disable SIP in recovery mode and disable Apple Mobile File Integrity (AMFI). Both of these actions are strongly discouraged, as they are mostly responsible for preventing you from running dtruss/strace.
• Your application must run as root.
• Your application must be granted 'Full Disk Access' from the 'Security & Privacy' System Preferences pane.

You could then write a complete replacement for strace as an application bundle (/Application/strace.app) and use it as such:

/Applications/ -e trace=execve -v -s 100000000 -xx -ttt -ff -o output.txt sh -c 'echo hi' 

The various suggestions in the other answers fail for different reasons:

• The use of dtrace now requires SIP to be disabled.
• If you could port strace to macOS, it would also require SIP to be disabled.
• Also in 13.2, execsnoop requires SIP:

% sudo execsnoop -a -c ./test
probe description proc:::exec-success does not match any probes. System Integrity Protection is on

My suggestion:

Rewrite the code to not use strace on either Linux or macOS and build a framework to get the information you need during the build directly. At some point Linux will add these type of security features. Maybe not this year, but I'd think before the end of the decade for sure.

Bonus:

If you wish to implement the application I suggested above, this gist is a start. Charles Duffy linked a derivative of that gist above.

You don't want to disable SIP with csrutil, it seems that the only option is to remove signature with codesign.

I made a little script to simplify the process, save following in dtruss.sh

#!/bin/bash file=$(type -P $1); shift c=/tmp/$(basename "$file") cp "$file" "$c" codesign --remove-signature "$c" sudo dtruss "$c" "$@" 

then

chmod +x dtruss.sh ./dtruss.sh echo hi ./dtruss.sh find /etc/ -name bashrc 

Update

$ ./dtruss.sh echo "Hello, World!" dtrace: system integrity protection is on, some features will not be available SYSCALL(args) = return Hello, World! mprotect(0x1105F0000, 0x8000, 0x1) = 0 0 thread_selfid(0x0, 0x0, 0x0) = 19299 0 shared_region_check_np(0x7FF7BAE92940, 0x0, 0x0) = 0 0 thread_selfid(0x0, 0x0, 0x0) = 19299 0 ... 

For the millionth time, I upgraded macOS to the next major version (Monterey) and I had to search for how to re-enable password-less sudo’ing (don’t judge me.)

edit /etc/sudoers

 sudo visudo 

Then find the admin group permission section:

 %admin ALL = (ALL) ALL 

Change to add NOPASSWD:

 %admin ALL = (ALL) NOPASSWD: ALL 

Profit until next year.

The short and simple answer is that the security model of MacOS is inherently different. The standard tool for tracing system calls (and much, much more) on BSD-based systems is dtruss, but it's quite different from strace; for starters, it requires you to be root to even start exploring what's possible.

Since it's not clear if that's a route you want to take, I will just briefly link to a few resources for further information.

So, in very brief, it is unlikely that you can easily extend this code to be portable between Linux and MacOS.

Perhaps a plausible alternative would be to extend the software you are attempting to strace to instead provide some sort of internal debugging mode to emit an event or print a message each time it dispatches an "interesting" system call. Depending on what's "interesting", this can be anywhere from moderately trivial to extremely challenging.

Another possible avenue to explore would be a wrapper for the system calls you are interested in. You could simply print out the arguments, then proceed to call the proper system call. See e.g. DYLD_LIBRARY_PATH & DYLD_INSERT_LIBRARIES not working (The equivalent facility on Linux is called LD_PRELOAD.)

Please check the following two scripts based on this book. I believe these two combined will give you a similar result as the original code.

I fixed the script to be compatible to new version. It runs on my M1. Not sure if I am getting the expected output. Please read the book for more information.

#openproc.py ( Fixed to newer_version) #!/usr/bin/env python import re import sys # ---------------------------- # openproc.py - Collect data from opentrace.py and merge :entry and :return → # Open trace file or use stdin try: inf = file(sys.argv[1], 'r') except OSError as ose: print(ose) print('''openproc.py [filename]''') sys.exit(1) except IndexError: inf = sys.stdin # Convert time to human time def human_time(ns): ns = float(ns) for unit in ['ns', 'us', 'ms']: if abs(ns) == 0: print('ERROR %d' % ret) else: print('OPEN %s %d %s => %s [%s]' % (users.get(uid, str(uid)), pid, state[pid][1], status, human_time(tm - state[pid][0]))) del state[pid] 

#opentrace.py (Fixed syntax) #!/usr/bin/env python import sys, os, subprocess, platform from optparse import OptionParser # ---------------------------- # opentrace.py - Trace open syscalls via SystemTap or DTrace # supports filtering per UID or PID optparser = OptionParser() optparser.add_option('-S', '--stap', action='store_true', dest='systemtap', help='Run SystemTap') optparser.add_option('-D', '--dtrace', action='store_true', dest='dtrace', help='Run DTrace') optparser.add_option('-p', '--pid', action='store', type='int', dest='pid', default='-1', metavar='PID', help='Trace process with specified PID') optparser.add_option('-u', '--uid', action='store', type='int', dest='uid', default='-1', metavar='UID', help='Filter traced processes by UID') optparser.add_option('-c', '--command', action='store', type='string', dest='command', metavar='CMD', help='Run specified command CMD and trace it') (opts, args) = optparser.parse_args() if opts.pid >= 0 and opts.command is not None: optparser.error('-p and -c are mutually exclusive') if (opts.pid >= 0 or opts.command is not None) and opts.uid >= 0: optparser.error('-p or -c are mutually exclusive with -u') if opts.systemtap and opts.dtrace: optparser.error('-S and -D are mutually exclusive') if not opts.systemtap and not opts.dtrace: # Try to guess based on operating system opts.systemtap = sys.platform == 'linux2' opts.dtrace = sys.platform == 'sunos5' if not opts.systemtap and not opts.dtrace: optparser.error('DTrace or SystemTap are non-standard for your platform,please specify -S or -D option') def run_tracer(entry, ret, cond_proc, cond_user, cond_default, env_bin_var, env_bin_path, opt_pid, opt_command, args, fmt_probe): cmdargs = [os.getenv(env_bin_var, env_bin_path)] if opts.pid >= 0: cmdargs.extend([opt_pid, str(opts.pid)]) entry['cond'] = ret['cond'] = cond_proc elif opts.command is not None: cmdargs.extend([opt_command, opts.command]) entry['cond'] = ret['cond'] = cond_proc elif opts.uid >= 0: entry['cond'] = ret['cond'] = cond_user % opts.uid else: entry['cond'] = ret['cond'] = cond_default cmdargs.extend(args) proc = subprocess.Popen(cmdargs, stdin=subprocess.PIPE) proc.stdin.write(fmt_probe % entry) proc.stdin.write(fmt_probe % ret) proc.stdin.close() proc.wait() if opts.systemtap: entry = {'name': 'syscall.open', 'dump': '''printf("=> uid: %d pid: %d open: %s %d\\n", uid(), pid(), filename, gettimeofday_ns());'''} ret = {'name': 'syscall.open.return', 'dump': '''printf(" uid: %%d pid: %%d open: %%s %%lld\\n", uid, pid, copyinstr(%s), (long long) timestamp); ''' % fn_arg} 

I have no experience in this, but I'll try my best.

You can use the DTrace.

Here are some commands from Oracle docs - DTrace command line:

dtrace [-CeFGhHlqSvVwZ] [-b bufsz] [-c command] [-D name[=value]] [-I pathname] [-L pathname] [-o pathname] [-p PID] [-s source_pathname] [-U name] [-x option[=value]][-X[a|c|s|t]] [-P provider[[predicate]action]] [-m [[provider:]module[[predicate]action]]] [-f [[provider:]module:]function[[predicate]action]] [-n [[[provider:]module:]function:]name[[predicate]action]] [-i probe-id[[predicate]action]] 

Now apparently this only operates in the D-language, but there are many githubs online that can handle this with python.

There is this particular Github by @paulross, where the preparation of creating a dtrace version of Python and a virtual environment in the ~/venvs directory (or wherever you prefer):

cd ~/tmp curl -o Python-3.7.0.tgz tar -xzf Python-3.7.0.tgz cd Python-3.7.0 ./configure --with-dtrace make python.exe -m venv ~/venvs/dtrace 

Unfortunately I cannot provide the exact code using dtrace that is used with strace due to my inexperience in this.

Here's an enormously long and comprehensive book on DTrace

See:

original:

# root and users in group wheel can run anything on any machine as any user root ALL = (ALL) ALL %admin ALL = (ALL) ALL 

change

 %admin ALL = (ALL) NOPASSWD: ALL 

ncG1vNJzZmirpJawrLvVnqmfpJ%2Bse6S7zGiorp2jqbawutJobmxvYml9eICOoaawZaSkeqSty6VkmqZdmr62tdWao56mpGKwsLnMmqWdZaSkerTA0ZqanmWfo3qurcJmoJ2dkaG5unnFq6amZaCuwam7zQ%3D%3D

 Share!