Top command-line Questions

List of Tags

Is there a (unix) shell script to format JSON in human-readable form?

Basically, I want it to transform the following:

{ foo: "lorem", bar: "ipsum" }

... into something like this:

    foo: "lorem",
    bar: "ipsum"


Answered By: B Bycroft ( 704)

With python you can just do

echo '{"foo": "lorem", "bar": "ipsum"}' | python -mjson.tool

I've got a rake task that I am making that needs to insert a value into multiple databases.

I'd like to be able to pass this value into the rake task from the command line, or from another rake task, how can I do this?

Answered By: Nick Desjardins ( 388)

You can specify formal arguments in rake by adding symbol arguments to the task call. For example:

require 'rake'

task :my_task, :arg1, :arg2 do |t, args|
  puts "Args were: #{args}"

task :invoke_my_task do
  Rake.application.invoke_task("my_task[1, 2]")

# or if you prefer this syntax...
task :invoke_my_task_2 do
  Rake::Task[:my_task].invoke(3, 4)

# a task with prerequisites passes its 
# arguments to it prerequisites
task :with_prerequisite, :arg1, :arg2, :needs => :my_task

# equivalently...
task :with_prerequisite_2, [:arg1, :arg2] => :my_task

# to specify default values, 
# we take advantage of args being a Rake::TaskArguments object
task :with_defaults, :arg1, :arg2 do |t, args|
  args.with_defaults(:arg1 => :default_1, :arg2 => :default_2)
  puts "Args with defaults were: #{args}"

Then, from the command line:

> rake my_task[1,2]
Args were: {:arg1=>"1", :arg2=>"2"}

> rake "my_task[1, 2]"
Args were: {:arg1=>"1", :arg2=>"2"}

> rake invoke_my_task
Args were: {:arg1=>"1", :arg2=>"2"}

> rake invoke_my_task_2
Args were: {:arg1=>3, :arg2=>4}

> rake with_prerequisite[5,6]
Args were: {:arg1=>"5", :arg2=>"6"}

> rake with_prerequisite_2[7,8]
Args were: {:arg1=>"7", :arg2=>"8"}

> rake with_defaults
Args with defaults were: {:arg1=>:default_1, :arg2=>:default_2}

> rake with_defaults['x','y']
Args with defaults were: {:arg1=>"x", :arg2=>"y"}

As demonstrated in the second example, if you want to use spaces, the quotes around the target name are necessary to keep the shell from splitting up the arguments at the space.

Looking at the code in rake.rb, it appears that rake does not parse task strings to extract arguments for prerequisites, so you can't do task :t1 => "dep[1,2]". The only way to specify different arguments for a prerequisite would be to invoke it explicitly within the dependent task action, as in :invoke_my_task and :invoke_my_task_2.


As I sometimes have path problems, where one of my own cmd scripts is hidden (shadowed) by another program (earlier on the path), I would like to be able to find the full path to a program in Windows, given just its name.

Is there an equivalent to the UNIX command 'which'?

On UNIX, which command prints the full path of the given command to easily find and repair these shadowing problems.

Answered By: Michael Ratanapintha ( 312)

Windows Server 2003 and later provide the WHERE command which does some of what which does, though it matches all types of files, not just executable commands. (It does not match built-in shell commands like cd.) It will even accept wildcards, so where nt* finds all files in your %PATH% and current directory whose names start with nt.

Try where /? for help.

Chris Serra

Is there a way to include all the jar files within a directory in the classpath?

I'm trying java -classpath lib/*.jar:. my.package.Program and it is not able to find class files that are certainly in those jars. Do I need to add each jar file to the classpath separately?

Answered By: basszero ( 192)

If using Java 6 or later, classpath wildcards are a part of the JVM.

java -cp "Test.jar;lib/*" my.package.MainClass

Key gotchas:

  1. Use quotes
  2. Use * only, not *.jar

The above example and gotchas are from other answers on this page. (Thanks davorp et al & Wim Deblauwe)

From the Classpath document section entitled, Understanding class path wildcards:

Class path entries can contain the basename wildcard character *, which is considered equivalent to specifying a list of all the files in the directory with the extension .jar or .JAR. For example, the class path entry foo/* specifies all JAR files in the directory named foo. A classpath entry consisting simply of * expands to a list of all the jar files in the current directory.

A class path entry that contains * will not match class files. To match both classes and JAR files in a single directory foo, use either foo;foo/* or foo/*;foo. The order chosen determines whether the classes and resources in foo are loaded before JAR files in foo, or vice versa.

Subdirectories are not searched recursively. For example, foo/* looks for JAR files only in foo, not in foo/bar, foo/baz, etc.

The order in which the JAR files in a directory are enumerated in the expanded class path is not specified and may vary from platform to platform and even from moment to moment on the same machine. A well-constructed application should not depend upon any particular order. If a specific order is required then the JAR files can be enumerated explicitly in the class path.

Expansion of wildcards is done early, prior to the invocation of a program's main method, rather than late, during the class-loading process itself. Each element of the input class path containing a wildcard is replaced by the (possibly empty) sequence of elements generated by enumerating the JAR files in the named directory. For example, if the directory foo contains a.jar, b.jar, and c.jar, then the class path foo/* is expanded into foo/a.jar;foo/b.jar;foo/c.jar, and that string would be the value of the system property java.class.path.

The CLASSPATH environment variable is not treated any differently from the -classpath (or -cp) command-line option. That is, wildcards are honored in all these cases. However, class path wildcards are not honored in the Class-Path jar-manifest header.

If you cannot use wildcards, bash allows the following syntax (where lib is the directory containing all the Java archive files):

java -cp $(echo lib/*.jar | tr ' ' ':')

(Note that using a classpath is incompatible with the -jar option. See also: Execute jar file with multiple classpath libraries from command prompt)


I'm looking for the string "foo=" (without quotes) in text files in a directory tree. It's on a common Linux machine, I have bash shell:

grep -ircl "foo=" *

In the directories are also many binary files which match "foo=". As these results are not relevant and slow down the search, I want grep to skip searching these files (mostly JPEG and PNG images): how would I do that?

I know there are the --exclude=PATTERN and --include=PATTERN options, but what is the pattern format? manpage of grep says:

--include=PATTERN     Recurse in directories only searching file matching PATTERN.
--exclude=PATTERN     Recurse in directories skip file matching PATTERN.

Searching on grep include, grep include exclude, grep exclude and variants did not find anything relevant

If there's a better way of grepping only in certain files, I'm all for it; moving the offending files is not an option, I can't search only certain directories (the directory structure is a big mess, with everything everywhere). Also, I can't install anything, so I have to do with common tools (like grep or the suggested find).

UPDATES: @Adam Rosenfield's answer is just what I was looking for:

grep -ircl --exclude=*.{png,jpg} "foo=" *

@rmeador's answer is also a good solution:

grep -Ir --exclude="*\.svn*" "pattern" *

It searches recursively, ignores binary files, and doesn't look inside Subversion hidden folders.(...)

Answered By: Adam Rosenfield ( 154)

Use the shell globbing syntax:

grep pattern -r --include=\*.{cpp,h} rootdir

The syntax for --exclude is identical.

Note that the star is escaped with a backslash to prevent it from being expanded by the shell (quoting it, such as --include="*.{cpp,h}", would work just as well). Otherwise, if you had any files in the current working directory that matched the pattern, the command line would expand to something like grep pattern -r --include=foo.cpp --include=bar.h rootdir, which would only search files named foo.cpp and bar.h, which is quite likely not what you wanted.

Alan Storm

I find working on the command line in Windows frustrating, primarily because the console window is wretched to use compared to terminal applications on linux and OS X such as "rxvt", "xterm", or "Terminal". Major complaints:

  1. No standard copy/paste. You have to turn on "mark" mode and it's only available from a multi-level popup triggered by the (small) left hand corner button. Then copy and paste need to be invoked from the same menu

  2. You can't arbitrarily resize the window by dragging, you need to set a preference (back to the multi-level popup) each time you want to resize a window

  3. You can only make the window so big before horizontal scroll bars enter the picture. Horizontal scroll bars suck.

  4. With the cmd.exe shell, you can't navigate to folders with \\netpath notation (UNC?), you need to map a network drive. This sucks when working on multiple machines that are going to have different drives mapped

Are there any tricks or applications, (paid or otherwise), that address these issue?

Answered By: Maximus ( 120)

Sorry for the self-promotion, I'm the author of another Console Emulator, not mentioned here.

ConEmu is opensource console emulator with tabs, which represents multiple consoles and simple GUI applications as one customizable GUI window.

Initially, the program was designed to work with Far Manager (my favorite shell replacement - file and archive management, command history and completion, powerful editor). But ConEmu can be used with any other console application or simple GUI tools (like PuTTY for example). ConEmu is a live project, open to suggestions.

A brief excerpt from the long list of options:

  • Latest versions of ConEmu may set up itself as default terminal for Windows
  • Use any font installed in the system, or copied to a folder of the program (ttf, otf, fon, bdf)
  • Run selected tabs as Administrator (Vista+) or as selected user
  • Windows 7 Jump lists and Progress on taskbar
  • Integration with DosBox (useful in 64bit systems to run DOS applications)
  • Smooth resize, maximized and fullscreen window modes
  • Scrollbar initially hidden, may be revealed by mouseover or checkbox in settings
  • Optional settings (e.g. pallette) for selected applications
  • User friendly text and block selection (from keyboard or mouse), copy, paste, text search in console
  • ANSI X3.64 and Xterm 256 color

Far Manager users will acquire shell style drag-n-drop, thumbnails and tiles in panles, tabs for editors and viewers, true colors and font styles (italic/bold/underline).

PS. Far Manager supports UNC paths (\\server\share\...)

Dan Fabulich

Xcode 3.2 provides an awesome new feature under the Build menu, "Build and Archive" which generates an .ipa file suitable for Ad Hoc distribution. You can also open the Organizer, go to "Archived Applications," and "Submit Application to iTunesConnect."

Is there a way to use "Build and Archive" from the command line (as part of a build script)? I'd assume that xcodebuild would be involved somehow, but the man page doesn't seem to say anything about this.

UPDATE Michael Grinich requested clarification; here's what exactly you can't do with command-line builds, features you can ONLY do with Xcode's Organizer after you "Build and Archive."

  1. You can click "Share Application..." to share your IPA with beta testers. As Guillaume points out below, due to some Xcode magic, this IPA file does not require a separately distributed .mobileprovision file that beta testers need to install; that's magical. No command-line script can do it. For example, Arrix's script (submitted May 1) does not meet that requirement.
  2. More importantly, after you've beta tested a build, you can click "Submit Application to iTunes Connect" to submit that EXACT same build to Apple, the very binary you tested, without rebuilding it. That's impossible from the command line, because signing the app is part of the build process; you can sign bits for Ad Hoc beta testing OR you can sign them for submission to the App Store, but not both. No IPA built on the command-line can be beta tested on phones and then submitted directly to Apple.

I'd love for someone to come along and prove me wrong: both of these features work great in the Xcode GUI and cannot be replicated from the command line.

Answered By: vdaubry ( 161)

I finally found how to automate the build and archive process from the comand line !! I just wrote a blog article explaining how you can achieve that.

The command you have to use is "xcrun" :

/usr/bin/xcrun -sdk iphoneos PackageApplication -v "${RELEASE_BUILDDIR}/${APPLICATION_NAME}.app" -o "${BUILD_HISTORY_DIR}/${APPLICATION_NAME}.ipa" --sign "${DEVELOPER_NAME}" --embed "${PROVISONING_PROFILE}"

You will find all the details in the article ! If you have any questions dont hesitate to ask.

Hope this helps, Vincent


How do I execute a command-line program from C# and get back the STD OUT results. Specifically, I want to execute DIFF on two files that are programmatically selected and write the results to a text box. Yes, I could figure this out for myself, but surely someone else has done something like it and I'm lazy...

Answered By: Ray Jezek ( 174)
// Start the child process.
 Process p = new Process();
 // Redirect the output stream of the child process.
 p.StartInfo.UseShellExecute = false;
 p.StartInfo.RedirectStandardOutput = true;
 p.StartInfo.FileName = "YOURBATCHFILE.bat";
 // Do not wait for the child process to exit before
 // reading to the end of its redirected stream.
 // p.WaitForExit();
 // Read the output stream first and then wait.
 string output = p.StandardOutput.ReadToEnd();

Code is from MSDN.


We all know how to use <ctrl>-R to reverse search through history, but did you know you can use <ctrl>-S to forward search if you set stty stop ""? Also, have you ever tried running bind -p to see all of your keyboard shortcuts listed? There are over 455 on Mac OS X by default.

What is your single most favorite obscure trick, keyboard shortcut or shopt configuration using bash?

Answered By: user10765 ( 158)

Renaming/moving files with suffixes quickly:
cp /home/foo/realllylongname.cpp{,-old}

This expands to:
cp /home/foo/realllylongname.cpp /home/foo/realllylongname.cpp-old

I needed to pass id and password to a cmd (or bat) file at the time of running rather than hardcoding them into the file.

Here's how I do it.

@echo off
fake-command /u %1 /p %2

Here's what the command line looks like:

test.cmd admin P@55w0rd > test-log.txt

The %1 applies to the first parameter the %2 (and here's the tricky part) applies to the second. You can have up to 9 parameters passed in this way.

Afterward: This is my first attempt to answer my own question which, to hear Jeff discuss it is a "...perfectly acceptable...." way of using SO. I'm just not certain if there's already a format for doing it.

Answered By: Keng ( 24)

Here's my answer in the new correct 'answer your own question' format.

Here's how I do it.

@fake-command /u %1 /p %2

Here's what the command line looks like:

test.cmd admin P@55w0rd > test-log.txt

The %1 applies to the first parameter the %2 (and here's the tricky part) applies to the second. You can have up to 9 parameters passed in this way.