Examples¶
As luci is designed to help with a fairly generic (and weird) pattern, I find it quite hard to describe what problems it can help you with. I figured, showcasing examples might make more sense. Even if – as you’ll probably discover – some of those examples might seem a bit far-fetched…
Metadata¶
Extract metadata from any textfile¶
(…as long as the file’s format allows for comments)
Say, you’ve got a bunch of csv files that you use for your research project. Now, the all contain ‘raw’ data, but you might want to ‘tag’ them with certain metadata fields to be able to process them in groups, or for whatever else reason. You can of course use the filename and a certain folder structure to organize them (which you should do in either case). But that can be fairly limited if there are overlapping categories.
So, why not add metadata directly into the data files (provided the file-format you use allows for comments or similar)? In that case luci can help you extract it. Of course, you could do that anyway and write your own script to get the metadata out. But why would you, now that luci exists, right?
Here’s how the csv file (let’s call it data.csv) could look (to be honest, I have no idea how to add comments to csv files, but let’s just assume we can use ‘#’ line-prefixes):
# lucify: start
# author: Markus Binsteiner
# date: "2018-03-03"
# tags:
# - luci
# - example
date,subject,result
2018-01-01,alice,okish
2018-01-02,tom,gone crazy
And here’s how to get the metadata out, using ‘json’ as output format:
❯ luci metadata --format json data.csv
{
"author": "Markus Binsteiner",
"date": "2018-03-03",
"tags": [
"luci",
"example"
]
}
You could use that output in a pipeline, possibly with jq_ or in a script.
Templating¶
Super-basic example¶
Say, we have a file containing:
# __luci__:
# vars:
# - he
# - she
What he said: {{ he }}
Then, she said: {{ she }}
Save that to a file named ‘who_said_what’. Users of luci can query this dictlet (this is what we call files that are consumed by luci) now for it’s interface:
❯ luci template who_said_what --help
Usage: luci template who_said_what [OPTIONS]
Options:
--he TEXT n/a
--she TEXT n/a
--help Show this message and exit.
And, to do the actual templating, we do something like:
❯ luci template who_said_what --he "I don't care" --she "I don't care either"
What he said: I don't care
Then, what she said then: I don't care either
This of course is not really useful, but those dictlets can become much more complex (using loops, conditional statements, etc.), as luci uses the powerful jinja2 as it’s templating engine.
Using default value for user arg¶
In the previous example we used the most basic user input options, 2 strings. luci supports much more specific options though. For example, say we want to add an optional name for the 2nd person saying stuff. We’d change the dictlet like so:
# __luci__:
# vars:
# - he
# - she
# - name:
# default: she
What he said: {{ he }}
Then, {{ name }} said: {{ she }}
Now, the --help options gives us:
❯ luci template who_said_what --help
Usage: luci template who_said_what_name
[OPTIONS]
Options:
--he TEXT n/a
--she TEXT n/a
--name TEXT
--help Show this message and exit.
And we can run it like:
❯ luci template who_said_what --he "Wha?" --she "Ya!" --name "Lola"
What he said: Wha?
Then, Lola said: Ya!
As luci uses the click Python library to dynamically create command-line linterfaces, we can use all of Click’s option features to control the created interface. More details on how to create a cli with luci can be found here.
Add boilerplate to a project¶
When developing, often you need to add boilerplate code to a file or project, with just a few strings different between each instance. Perfect use-case for templating. Let’s use luci itself as an example. luci uses what is called a lucifier to implement different types of tasks that use dictlet metadata and content as input. Those all inherit from a base class, and need to contain a few attributes and functions. We can easily use a template, combined with user input, to either attach source code describing such a class to a file (or just create a new file). Here’s how that particular ‘add-lucifier’ dictlet looks like:
#! /usr/bin/env luci
#
# __lucify__:
# __default_lucifier__: template
# vars:
# lucifier.name:
# meta:
# type: str
# required: true
# doc:
# help: the name of the lucifier (without the 'Lucifier'-postfix)
# cli:
# option:
# nargs: 1
# required: true
# param_decls: ["--name", "-n"]
# lucifier.help:
# meta:
# type: str
# required: false
# alias: lucifier-help
# doc:
# help: help string for this lucifier
# lucifier.short_help:
# meta:
# type: str
# required: false
# alias: lucifier-short-help
# doc:
# help: short help string for this lucifier
# lucifier.epilog:
# meta:
# type: str
# required: false
# alias: lucifier-epilog
# doc:
# help: epilog string for this lucifier
# lucifier.var_names:
# meta:
# type: list
# required: false
# doc:
# help: names of cli variable names
# cli:
# option:
# param_decls: ["--var-names", "-v"]
# multiple: true
# type: str
class {{ lucifier.name | capitalize }}Lucifer(Lucifier):
"""{{ lucifier.help }}
"""
LUCI_{{ lucifier.name | upper }} = {
"doc": {
"short_help": "{{ lucifier.short_help }}",
# "epilog": "{{ lucifier.epilog }}"
},
"vars": {
{% for var_name in lucifier.var_names %}"__{{ lucifier.name | lower }}__.{{ var_name }}": {
"type": str,
"required": False,
"default": False,
"doc": {
"help": "<var help for {{ var_name }}>"
},
"click": {
# check http://click.pocoo.org/6/api/#parameters for details and available keys
"option": {
"param_decls": ["--{{ var_name }}"],
"type": str,
# potential other options ...
},
# "argument": {
# "param_decls": '{{ var_name }}',
# "nargs": 1
}
}
},
{% endfor -%}
# if you want the following two parameters, you need a: from luci import utils somewhere in this file
# "pager": utils.create_pager_var(),
# "output_type": lucifiers.create_output_type_var()
}
}
def __init__(self, **kwargs):
super({{ lucifier.name | capitalize }}, self). __init__(**kwargs)
def get_default_metadata(self):
return {{ lucifier.name | capitalize }}Lucifier.LUCI_{{ lucifier.name | upper }}
def process_dictlet(self, metadata, dictlet_details):
utils.output(metadata, format="raw", pager=False)
Note how we use click - specific parameters param_decls and multiple for the lucifier.var_names variable:
# lucifier.var_names:
# meta:
# type: list
# required: false
# doc:
# help: names of cli variable names
# cli:
# option:
# param_decls: ["--var-names", "-v"]
# multiple: true
# type: str
To output the assembled boilerplate template to stdout, all we need to do is:
❯ luci template add-lucifier --help
Usage: luci template add-lucifier [OPTIONS]
Options:
-n, --name TEXT the name of the lucifier (without the
'Lucifier'-postfix) [required]
--lucifier-help TEXT help string for this lucifier
--lucifier-short-help TEXT short help string for this lucifier
--lucifier-epilog TEXT epilog string for this lucifier
-v, --var-names TEXT names of cli variable names
--help Show this message and exit.
❯ luci template add-lucifier --name example --lucifier-short-help "An example lucifier" --lucifier-help "An example lucifier" -v input_var_1 -v input_var_2
class ExampleLucifer(Lucifier):
"""An example lucifier
"""
LUCI_EXAMPLE = {
"doc": {
"short_help": "An example lucifier",
# "epilog": ""
},
"vars": {
"__example__.input_var_1": {
"type": str,
"required": False,
"default": False,
"doc": {
"help": "<var help for input_var_1>"
},
...
... <more boilerplate>
...
Download¶
Include update information into a shell script¶
Say, we wrote a cool bash-script we are really proud of:
#!/usr/bin/bash
#
# Copyright Markus Binsteiner, 2018
# version 0.1
echo "Awesome!"
Now, somebody found a bug in the script. So, we fix that bug, and we release a new version. How do we get that new version to our users easily, given that it’s only a shell script, not packaged. And a git checkout of a whole repository doesn’t make all that much sense for only one file either?
With luci, we can add the remote url of the script in a commented section somewhere, like:
#!/usr/bin/env bash
#
# lucify: start
#
# url:
# default: https://raw.githubusercontent.com/makkus/luci-examples/master/download/awesome/awesome.sh
# version: https://raw.githubusercontent.com/makkus/luci-examples/{{ version }}/download/awesome/awesome.sh
#
# lucify: stop
#
# Copyright Markus Binsteiner, 2018
# version 0.2
echo "Super awesome!"
Then, if one wanted to update the script itself, they’d issue:
❯ luci download awesome.sh --replace
Downloading: https://raw.githubusercontent.com/makkus/luci-examples/master/download/awesome/awesome.sh
Saving file: /home/markus/.local/bin/awesome.sh
Done.
The --replace option is necessary, as otherwise the download command would only print out the content of the remote file. To see all the available options for a particular task, we can use the --help flag:
❯ luci download awesome.sh --help
Usage: luci download awesome.sh [OPTIONS]
Options:
-r, --replace if set, the target file will be overwritten
-t, --target PATH if specified, the url content will be saved into the specified file,
otherwise the dictlet itself will be replaced
--version VERSION the version of the dictlet to download (optional)
--help Show this message and exit.
Of course, this example only makes limited sense, as we’d have to ask our users to install a Python package (luci) just to update a single bash script. Let’s ignore that though, shall we?
Scripting¶
Executable dictlets¶
You can make every dictlet (that is not a script by itself in the first place) executable by setting luci as it’s interpreter in the she-bang line, and adding the __default_lucifier__ key in the __luci__ metadata:
#! /usr/bin/env luci
#
# __luci__:
# __default_lucifier__: template
# vars:
# - he
# - she
What he said: {{ he }}
Then, what she said: {{ she }}
Make this file executable (chmod +x who_said_what), put it somewhere in your $PATH and you’ve got a dynamic text emitter script:
❯ who_said_what --help
Usage: luci dev_input [OPTIONS]
Options:
--he TEXT n/a
--she TEXT n/a
--help Show this message and exit.
❯ who_said_what --he Right --she Ok
What he said: Right
Then, what she said: Ok