===============================================
fIcy: an icecast/shoutcast stream grabber suite
===============================================

.. contents::

fIcy is a small icecast/shoutcast stream grabber suite for use under shell
environment. Its goal is to automatically rip a stream into user customisable
files. It will work with ICY compatible streams, allowing you to either to save
the stream to disk or to pipe the output to a media player, or even both. fIcy,
among other uses, is ideal for batch/unattended recording of radio programs and
stream debugging.

The fIcy package includes:

* fIcy itself, a stream separator/multiplexer,
* fResync, a fast MPEG-resyncing utility,
* fPls, a playlist frontend for fIcy.


Usage
-----

Synopsis:

  ``fIcy [options] <server [port [path]]|url>``

    The main program. Takes directly a stream url and dumps the tracks on the
    specified file/s and standard output, depending on the settings.

  ``fPls [options] <file|url> [fIcy options]``

    Playlist manager. Reads a playlist (local or remote) and manages fIcy
    retries/timeouts/errors, forwarding the specified flags.

  ``fResync [options] file``

    MPEG resyncing utility. Re-aligns head frame headers on dumped or broken
    files. Usually needed for embedded hardware decoders or editing software.


fIcy options
~~~~~~~~~~~~

  -d		Do not dump the output to stdout.
		Useful when only ripping.

  -E num	Enumerate files when song title [metadata] changes, starting at
		num. When 0, fIcy will try to find the highest unused file
		number automatically. Uses -o as a prefix.

  -h		Help

  -c		Do not clobber files.

  -A		Append to existing files instead of overwriting.

  -m		Use song title [metadata] when naming files. Uses -o as	prefix.

  -n		If the file exists create new one with .n as suffix.

  -p		When dumping to stdout consider writing errors as transient
		(that is: flush the output buffer until stdout is ready).
		Useful when you pipe the output to a media player and want to
		kill it while not interrupting the rip.

  -o file	Dump the output to file or use file as a prefix (depending on
		other settings). Hint: to dump without a prefix use "./".

  -s suffix	Use sfx as a suffix for new files.
		Hint: the .mp3/.m4a extension is NOT implicit.

  -t		Display song title [metadata] while ripping.

  -r		Remove/don't save partial chunks. This will skip the first
		chunk and remove the last one upon termination which are
		(supposedly) incomplete. To use in combination with -m or -E.

  -q file	Append "file name" sequence to file. The file name is written
		upon file completition. This may be used to trigger events and
		rejoin splitted parts with an external tool without -E.
		fResync will use this file in the future.

  -x regex	Save only files whose title (NOT filename) matches against this
		(or one of these) extended regular expressions. Multiple -x can
		be specified on the command line to form OR conditions. Dump
		unaffected.  Can be combined with -X.

  -X regex	Do NOT save files whose title matches against this extended
		regular expression. Same semantics as -x.

  -I file	Load include/exclude REs from file. Each line must be prefixed
		with + or - to indicate whether it's a positive or negative
		expression (-xX).

  -f expr 	Filter titles through the specified coprocessor expression. The
		raw title is passed to the expression (doesn't include any
		additional prefixes/suffixes). As the result will be used
		internally, some limitations apply. Read carefully the
		Filtering_ section.

  -F file	Filter titles through the specified coprocessor script. Same
		semantics as -f, but the expressions are loaded from a file
		instead. Conflicts with -f.

  -C path	Specify the path of the external title rewriting coprocessor.
		Defaults to "sed". The executable must support the '-e', '-f'
		flags and operate through stdin/out, like "sed".

  -M time	Maximum recording time. See Notes_.

  -i time	Maximum network idle time. Stops recording after the specified
		amount of time is passed without network activity. Defaults to
		0 (default tcp timeout).

  -a file	Read authentication credentials from file (the file must
		contain a line of the form user:password). Note that only the
		Basic HTTP authentication scheme is supported.

  -l num	Redirect follow limit. Defaults to 1. 0 disables redirection
		entirely.


fResync options
~~~~~~~~~~~~~~~

  -b		By default fResync maps the entire file into memory when
		operating. However this can create problems on loaded systems
		with large files or when simulating. This reverts to a
		buffered I/O mode. This flag is also implicit when simulating.

  -s		Simulate the process. Print on the standard output the starting
		sync offset and stream length, but don't modify the source
		file.

  -v		Verbose.

  -n frames	Require/decode at least n valid consecutive frames to validate
		the sync offset. Defaults to 6.

  -m len	Maximum frame length. Defaults to 1597. fResync uses this value
		to determine the maximal region of the file to be checked.


fPls options
~~~~~~~~~~~~

  -P path	Specify a different name or full path for the fIcy executable
		(defaults to "fIcy").

  -v		Verbose.

  -R max	Specifies the maximal number of retries to do for each stream
		upon connection/read failure.

  -L max	Specifies the maximal number of loops to do for the entire
		playlist (-1 for infinite).

  -T time	Wait time to pause after each failure.

  -M time	Maximum cumulative recording time. See Notes_.

  -i time	Maximum network idle time. Same as fIcy's when loading a
		playlist via http. Forwarded to fIcy.

  -a file	Read authentication credentials from file. Same as fIcy's when
		loading a playlist via http. The credentials are automatically
		forwarded to fIcy, but you can override them when needed.

  -l num	Redirect follow limit. Same as fIcy's when loading a playlist
		via http. Forwarded to fIcy.

  -d file	Run as a daemon, redirecting messages to file. fIcy's -d
		option is enforced. As the process is chdir-ed to the root
		directory you also have to specify absolute paths for all
		options, including fIcy's ones.


Examples
~~~~~~~~

Rip a station until stopped::

	mkdir files
	fPls -L-1 http://netradio.invalid/listen.pls -s .mp3 -o files/ -cmrdt

Use fIcy to manage reconnections and display titles while playing::

	fPls http://example.com:8080/listen.pls -t | mpg123 -

Connect directly to the stream with server:port and /path::

	fIcy -s .mp3 -o ./ -md 123.123.123.123 8080 /path/to/stream
	
Rip an .mp3 stream while playing, but allows the player to be restarted later
by using a named fifo (note that you can re/open "fifo" with any player)::

	$ mkfifo fifo
	$ fIcy -p ... > fifo
	$ mpg123 fifo

Record your favourite program "XYZ" usually on-air between 16:30-17:00::

	at 16:30
	fPls -M 30m http://example.com/listen.pls -o program.mp3 -x XYZ
	^D

Cleanup a ripped and/or damaged mp3 file::

	fResync file.mp3


Companion software
------------------

`bfr <http://www.glines.org/software/bfr>`_:

	Audio-oriented rebuffering tool. Ideal for lousy streams.

`mpgedit <http://www.mpgedit.org/>`_:

	Frame-level mp3 cutting tool.


DISCLAIMER
----------

We would like to remind you that saving streams containing copyrighted material
without explicit consent is *ILLEGAL*. For stream administrators, please see
our statement in the FAQ_.

Notes
-----

The output files produced by fIcy may miss audio framing information and
headers since the separation does not consider the audio data. For this reason,
your player 'may' (but should not) fail to reproduce the dump or output some
initial noise: this is expected. fResync can be used to cleanup MPEG files
after processing.

You can also use other tools such as mpgedit for cutting the file in arbitrary
positions without diminishing the quality. Assuming that your song spans across
three files (use -q to know which ones), that's how to proceed::

	cat 1.mp3 2.mp3 3.mp3 > temp.mp3 && xmpgedit temp.mp3

Do *not* resync the files if you're going to post-process them this way:
fResync would remove at least one boundary frame on each file, while other
tools could also insert extra empty frames to silence the decoder!

The -M flag supported by both fIcy and fPls accepts a time specification in
seconds, `HH:MM` or `N minutes/hours/days`. Also beware that -M specified in
fPls means `cumulative recording time` (time accumulates across
retries/timeouts), while -M specified in fIcy means `single stream recording
time` (recording stops at the first error or when the specified time has
elapsed).


Filtering
---------

Most online radio stations tend to put banners in the title that will be shown
in the player, and eventually result in the filename. To overcome to this (and
more), fIcy offers the possibility to rewrite each title through a normal sed
script via the "-fF" flags. A real sed coprocess is used along the execution so
all of sed's power is available, but some limitations apply:

* Each line of input should result in one output line, and ONE ONLY.

* Two consecutive identical titles will result in the second one
  being ignored (thus NOT splitting the stream). Consider this rule,
  as removing carefully the banner could result in a better separation.

* The resulting title will still apply for -xXI as usual.

* Please note that the *title* is filtered, not the filename (which may
  still have some characters removed/modified). Use -tv to see what is
  actually sent to the filter.

You can actually use any executable that works as a stream editor by specifying
the path with '-C'. The executable must support the '-e' (inline expression)
and '-f' (script file) flags or, at least, ignore them. This allows for any
script or custom executable to be used when a "sed" script is considered
inadequate.


Filtering examples
~~~~~~~~~~~~~~~~~~

As an example, suppose your titles look like this::

  Artist - Title (radiobanner)

You can write a sed expression or script containing::

  s/ (radiobanner)$//

to remove the trailing part. This facility can also be used to uniform file
names, invert Artist/Title positions and so on. Clever use of the pattern space
can also be used to merge albums. sed alone can be used to debug expressions,
eg::

  echo "test title" | sed -e 'expr'

Refer to the sed(1) manual for a complete list of commands you can use.


Installation
------------

fIcy comes with a very simple Makefile that should work on any system using gcc
and GNU make. Documentation is generated from these files using rst2html_.

All standard environment flags are supported, including DESTDIR/PREFIX for
relocated installation. If you need to use a different compiler (for example on
OpenBSD), you can call make as follows::

  make CXX=eg++ PREFIX=/usr install

instead of changing manually the Makefile. Please note that, when using gcc, at
least g++ >= 3 is required to compile fIcy.

Also note that, for "fPls" to work, fIcy must be already installed (be in
"PATH") or a full fIcy path must be specified with -P.

.. _rst2html: http://docutils.sourceforge.net/


General/support mailing list
----------------------------

If you feel to discuss improvements and suggestions, and/or test beta releases
before announcement you can subscribe to `ficy-users` by either sending an
empty email to <ficy-users+subscribe@thregr.org>, using GMane_ (group
"gmane.comp.audio.ficy.users") or by contacting the author at
<wavexx@thregr.org>. The archives are accessible via web through
https://www.mail-archive.com/ficy-users@thregr.org/ or via news directly.

.. _GMane: http://www.gmane.org/


Authors and Copyright
---------------------

"fIcy" can be found at https://www.thregr.org/~wavexx/software/fIcy/

| "fIcy" is distributed under GNU LGPLv2+, WITHOUT ANY WARRANTY.
| Copyright(c) 2004-2017 by Yuri D'Elia <wavexx@thregr.org>.

fIcy's GIT repository is publicly accessible at::

  git://src.thregr.org/fIcy

or at https://github.com/wavexx/fIcy
