0

Set operations in the unix shell are possible:

http://www.catonmat.net/blog/set-operations-in-unix-shell/

And the solutions from this site are very user-friendly. With "user" being a sub set of all people loving unix :-)

I search a library to do things like this:

set-ops fileA - fileB

With "fileX" being a set of lines in a file.

I want a easy to use tool:

  • The tool can use some kind of sorting for the implementation, but me (the user) don't want to call it.
  • I don't want to call "comm -3" or other things which are hard to remember.

This results into the restriction that the set members are strings which must not contain the newline character. But that's ok.

Needed features:

  • open source
  • easy to install (for me this means: rpm/dpkg/pip package)
  • available on linux.
  • Above link has a list of supported set operations (Set Membership to Maximum). Most of them should be implemented.
Improve this question
2
  • You want to do everything within the shell?
    – Bill Bell
    Nov 14, 2016 at 19:23
  • @BillBell what is "shell"? If I call a command like grep or sed, then there is not big difference to calling a special command, or a python/perl script. Short answer: I want to call the set operations tool via shell. I don't care at all about the implementation (it can be python, perl, c, go, ....). Dear Bill, did this answer your question?
    – guettli
    Nov 15, 2016 at 8:52

3 Answers 3

Reset to default
2
+25

I wonder why you did not mention his other post: http://www.catonmat.net/blog/set-operations-in-unix-shell-simplified/ that actually has the list of most set operations - the the scripts are available for download as .txt.

Improve this answer
2
  • Have you looked at the text file? It does not provide commands for the shell. I think it is just an other format (text instead of html).
    – guettli
    Nov 15, 2016 at 15:09
  • Yeah, I know. The question is - is this the sort of thing you are looking for or you want a custom script library, where say: set_union file1 file2 ... filen executes cat set1 set2 ... setn and say set_min set fires away head -1 <(sort set)?
    – Lefty G Balogh
    Nov 15, 2016 at 15:37
1

I wrote a python script to make set operations available on the command line.

Source Code is here: https://github.com/guettli/reprec/blob/master/setops/init.py

Since I wanted it to be ready before this bounty finishes, the source is in the repo of my reprec tool.

You can install it via pip: https://pypi.python.org/pypi/reprec

Usage:

===> setops -h
usage: setops [-h] set1 operator set2

Operators: 
  union Aliases: | + or
  intersection Aliases: & and
  difference Aliases: - minus
  symmetric_difference Aliases: ^

positional arguments:
  set1
  operator
  set2

optional arguments:
  -h, --help  show this help message and exit
Improve this answer
1
0

I might as well say what I was thinking when I commented! :)

Have you considered using Sage? It has a fairly complete collection of set operations, and also allows Python code within itself. If the text files you contemplate manipulating are not too large then you could write a quite small routine to convert a text file to a Sage set then use the Sage operators to manipulate instances of these sets.

Improve this answer
3
  • If I have two files like in the question text (fileA, fileB). The content is newline separated filenames. How can I do set operations on the shell with Sage?
    – guettli
    Nov 15, 2016 at 15:35
  • You can write Sage scripts that are callable from the shell. See doc.sagemath.org/html/en/tutorial/….
    – Bill Bell
    Nov 15, 2016 at 15:43
  • Hmm Sage is GPL. I am afraid that I need to open source my scripts which use Sage, since the GPL is a viral licence.
    – guettli
    Nov 15, 2016 at 16:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.