Overview of pyln-client implementation - Plugin.run() - Part 1
In this episode, we look a pyln-client Python package implementation focusing specifically on the method run of the class Plugin.
Transcript with corrections and improvements
Install pyln-client
Let's install pyln-client in .venv Python virtual
environment:
◉ tony@tony:~/lnroom:
$ python -m venv .venv
◉ tony@tony:~/lnroom:
$ source .venv/bin/activate
(.venv) ◉ tony@tony:~/lnroom:
$ pip install pyln-client
...
Setup
Here is my setup
(.venv) ◉ tony@tony:~/lnroom:
$ ./setup.sh
Ubuntu 22.04.2 LTS
Python 3.10.6
lightningd v23.02.2
pyln-client 23.2
Start 2 Lightning nodes running on regtest
Let's start two Lightning nodes running on the Bitcoin regtest chain
by sourcing the script lightning/contrib/startup_regtest.sh provided
in CLN repository and by running the command start_ln:
(.venv) ◉ tony@tony:~/lnroom:
$ source lightning/contrib/startup_regtest.sh
lightning-cli is /usr/local/bin/lightning-cli
lightningd is /usr/local/bin/lightningd
Useful commands:
start_ln 3: start three nodes, l1, l2, l3
connect 1 2: connect l1 and l2
fund_nodes: connect all nodes with channels, in a row
stop_ln: shutdown
destroy_ln: remove ln directories
(.venv) ◉ tony@tony:~/lnroom:
$ start_ln
Bitcoin Core starting
awaiting bitcoind...
Making "default" bitcoind wallet.
[1] 697735
[2] 697769
WARNING: eatmydata not found: instal it for faster testing
Commands:
l1-cli, l1-log,
l2-cli, l2-log,
bt-cli, stop_ln, fund_nodes
(.venv) ◉ tony@tony:~/lnroom:
We can check that l1-cli is just an alias for lightning-cli with the
base directory being /tmp/l1-regtest:
(.venv) ◉ tony@tony:~/lnroom:
$ alias l1-cli
alias l1-cli='lightning-cli --lightning-dir=/tmp/l1-regtest'
myplugin.py
To demonstrate how Plugin.run method works, we'll use myplugin.py
plugin. This plugin registers the JSON-RPC method node-id to
lightningd. That method returns the node id of the node running the
plugin:
#!/usr/bin/env python
from pyln.client import Plugin
plugin = Plugin()
@plugin.method("node-id")
def node_id_func(plugin):
node_id = plugin.rpc.getinfo()["id"]
return {"node_id":node_id}
plugin.run()
If you don't know how to write Core Lightning plugins with
pyln-client you can check Start writing Core Lightning plugins with
pyln-client TODAY!.
Let's jump in our terminal, start myplugin.py plugin
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.py
{
"command": "start",
"plugins": [...]
}
and call node-id method:
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli node-id
{
"node_id": "03fd248f93b8352d6cfc6a168f0fd10b1462f72d84b310a56f5c18076c5cf04fe4"
}
Part of Plugin.run() "call stack"
The method Plugin.run() is an IO loop waiting for incoming JSON-RPC
requests from lightningd.
Those incoming requests can be "normal" requests or notifications.
When an incoming request is received, it is passed to
Plugin._multi_dispatch method.
And in the case of "normal" requests (what we are studying today), the
method Plugin._multi_dispatch passes the request to
Plugin._dispatch_request method.
In Plugin._dispatch_request:
Plugin._exec_funcmethod constructs the payload (what goes into theresultfield of the JSON-RPC response) corresponding to the method of the request andRequest.set_resultproduces the JSON-RPC response to the request and passes it toRequest._write_resultmethod.
Then, Request._write_result replies to lightningd by writing to the
plugin's stdout stream.
Finally, we are back in the IO loop waiting for incoming JSON-RPC
request from lightningd.
What we've just described can be represented with the following schema
where the methods in the boxes are the methods we are going to modify
in that video to better understand how Plugin.run method works:
┌────────────┐
│Plugin.run()│
└┬───────────┘
└── Plugin._multi_dispatch
│ ┌────────────────────────┐
└──│Plugin._dispatch_request│
└┬───────────────────────┘
├── Plugin._exec_func
│ ┌──────────────────┐
└──│Request.set_result│
└┬─────────────────┘
└── Request._write_result
└── Plugin._write_locked
└── (write to Plugin.stdout)
Plugin.run
The method Plugin.run is defined in lightning:contrib/pyln-client/pyln/client/plugin.py like this:
class Plugin(object):
...
def run(self) -> None:
# If we are not running inside lightningd we'll print usage
# and some information about the plugin.
if os.environ.get('LIGHTNINGD_PLUGIN', None) != '1':
return self.print_usage()
partial = b""
for l in self.stdin.buffer:
partial += l
msgs = partial.split(b'\n\n')
if len(msgs) < 2:
continue
partial = self._multi_dispatch(msgs)
The requests received by run method are stored in the first field of
msgs array which is then passed to Plugin._multi_dispatch.
Let's adds the following Python snippet
with open("/tmp/myplugin_out", "a") as output:
output.write(f"\n--------\n---> in 'Plugin.run'\n{repr(msgs)}\n")
to Plugin.run (in the file plugin.py in .venv virtual environment)
like this
class Plugin(object):
...
def run(self) -> None:
# If we are not running inside lightningd we'll print usage
# and some information about the plugin.
if os.environ.get('LIGHTNINGD_PLUGIN', None) != '1':
return self.print_usage()
partial = b""
for l in self.stdin.buffer:
partial += l
msgs = partial.split(b'\n\n')
if len(msgs) < 2:
continue
with open("/tmp/myplugin_out", "a") as output:
output.write(f"\n--------\n---> in 'Plugin.run'\n{repr(msgs)}\n")
partial = self._multi_dispatch(msgs)
in order to see how does the requests look like by writing each
incoming request to the file /tmp/myplugin_out.
Back to our terminal we restart our plugin in order to take into
account the changes we made in pyln-client package:
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.py
{
"command": "stop",
"result": "Successfully stopped myplugin.py."
}
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.py
{
"command": "start",
"plugins": [...]
}
Now we can check that the file /tmp/myplugin_out contains the
getmanifest and init requests sent by lightningd to myplugin.py plugin
after we started it:
--------
---> in 'Plugin.run'
[b'{"jsonrpc":"2.0","id":96,"method":"getmanifest","params":{"allow-deprecated-apis":false}}', b'']
--------
---> in 'Plugin.run'
[b'{"jsonrpc":"2.0","id":"cln:init#97","method":"init","params":{"options":{},"configuration":{"lightning-dir":"/tmp/l1-regtest/regtest","rpc-file":"lightning-rpc","startup":false,"network":"regtest","feature_set":{"init":"08a000080269a2","node":"88a000080269a2","channel":"","invoice":"02000000024100"}}}}', b'']
Then we can call node-id method like this
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli node-id
{
"node_id": "03fd248f93b8352d6cfc6a168f0fd10b1462f72d84b310a56f5c18076c5cf04fe4"
}
and observe that node-id request has been written in the file
/tmp/myplugin_out which is now:
--------
---> in 'Plugin.run'
[b'{"jsonrpc":"2.0","id":96,"method":"getmanifest","params":{"allow-deprecated-apis":false}}', b'']
--------
---> in 'Plugin.run'
[b'{"jsonrpc":"2.0","id":"cln:init#97","method":"init","params":{"options":{},"configuration":{"lightning-dir":"/tmp/l1-regtest/regtest","rpc-file":"lightning-rpc","startup":false,"network":"regtest","feature_set":{"init":"08a000080269a2","node":"88a000080269a2","channel":"","invoice":"02000000024100"}}}}', b'']
--------
---> in 'Plugin.run'
[b'{ "jsonrpc" : "2.0", "method" : "node-id", "id" : "cli:node-id#698621/cln:node-id#108", "params" :[ ] }', b'']
Plugin._muti_dispatch
The method Plugin._multi_dispatch is defined in lightning:contrib/pyln-client/pyln/client/plugin.py like this:
class Plugin(object):
...
def _multi_dispatch(self, msgs: List[bytes]) -> bytes:
"""We received a couple of messages, now try to dispatch them all.
Returns the last partial message that was not complete yet.
"""
for payload in msgs[:-1]:
# Note that we use function annotations to do Millisatoshi
# conversions in _exec_func, so we don't use LightningJSONDecoder
# here.
request = self._parse_request(json.loads(payload.decode('utf8')))
# If this has an 'id'-field, it's a request and returns a
# result. Otherwise it's a notification and it doesn't
# return anything.
if request.id is not None:
self._dispatch_request(request)
else:
self._dispatch_notification(request)
return msgs[-1]
From msgs, Plugin._multi_dispatch instantiates request object of type
Request, check that the request has an id (which is the case of node-id
request for instance) and passes it to Plugin._dispatch_request (we
don't look at notifications in this video).
As it might be useful to see how Request class is defined we reproduce below its Request.__init__ method:
class Request(dict):
"""A request object that wraps params and allows async return
"""
def __init__(self, plugin: 'Plugin', req_id: Optional[str], method: str,
params: Any, background: bool = False):
self.method = method
self.params = params
self.background = background
self.plugin = plugin
self.state = RequestState.PENDING
self.id = req_id
self.termination_tb: Optional[str] = None
Plugin._dispatch_request
In Plugin._dispatch_request method, we look for the name of the
request in Plugin.methods dictionary. If we find it, we set method
variable as the object of the class Method associated with that name
in Plugin.methods. Then we try to execute the function method.func
using Plugin._exec_func and set result variable to the value returned
by Plugin._exec_func. If there is no error, we finally reply to
lightningd passing result to Request.set_result method:
class Plugin(object):
...
def _dispatch_request(self, request: Request) -> None:
name = request.method
if name not in self.methods:
raise ValueError("No method {} found.".format(name))
method = self.methods[name]
request.background = method.background
try:
result = self._exec_func(method.func, request)
if not method.background:
# Only if this is a synchronous (background=False) call do we need to
# return the result. Otherwise the callee (method) will eventually need
# to call request.set_result or request.set_exception to
# return a result or raise an exception.
request.set_result(result)
except Exception as e:
if name in hook_fallbacks:
response = hook_fallbacks[name]
self.log((
"Hook handler for {name} failed with an exception. "
"Returning safe fallback response {response} to avoid "
"crashing the main daemon. Please contact the plugin "
"author!"
).format(name=name, response=response), level="error")
request.set_result(response)
else:
request.set_exception(e)
self.log(traceback.format_exc())
To have a better understanding of the composition of Plugin.methods
dictionary, we can send the following expressions
from pyln.client import Plugin
plugin = Plugin()
to a Python interpreter and then looks at plugin.methods value:
>>> plugin.methods
{'init': <pyln.client.plugin.Method object at 0x7f03545d6080>,
'getmanifest': <pyln.client.plugin.Method object at 0x7f03545d5ed0>}
>>> plugin.methods["init"]
<pyln.client.plugin.Method object at 0x7f03545d6080>
>>> plugin.methods["init"].name
'init'
>>> plugin.methods["init"].func
<bound method Plugin._init of <pyln.client.plugin.Plugin object at 0x7f03545d7d60>>
It means that when myplugin.py receives the init request from
lightningd, the function used in Plugin._dispatch_request to build the
result field of the JSON-RPC reponse is Plugin._init method.
Now if we send the following Python decorator snippet
@plugin.method("node-id")
def node_id_func(plugin):
node_id = plugin.rpc.getinfo()["id"]
return {"node_id":node_id}
to our Python interpreter and look at plugin.methods value
>>> plugin.methods
{'init': <pyln.client.plugin.Method object at 0x7f03545d6080>,
'getmanifest': <pyln.client.plugin.Method object at 0x7f03545d5ed0>,
'node-id': <pyln.client.plugin.Method object at 0x7f0353423cd0>}
we can see that an entry for node-id method has been added. And if we
look at
>>> plugin.methods["node-id"].func
<function node_id_func at 0x7f03534315a0>
we can see that when myplugin.py receives a node-id request from
lightningd, the function used in Plugin._dispatch_request to build the
result field of the JSON-RPC reponse is node_id_func function we
defined just below the line @plugin.method("node-id").
How does this work? Well, if you are intereted you can check Overview of pyln-client implementation - @plugin.method() - Part 2.
Back to lightning:contrib/pyln-client/pyln/client/plugin.py file and Plugin._dispatch_request method.
Let's add the following Python snippet
with open("/tmp/myplugin_out", "a") as output:
output.write(f"---> in 'Plugin._dispatch_request\n{name}\n"+
f"{repr(method.func)}\n"+
f"{json.dumps(result)}\n")
to Plugin._dispatch_request (in the file plugin.py in .venv virtual
environment) like this
class Plugin(object):
...
def _dispatch_request(self, request: Request) -> None:
name = request.method
if name not in self.methods:
raise ValueError("No method {} found.".format(name))
method = self.methods[name]
request.background = method.background
try:
result = self._exec_func(method.func, request)
with open("/tmp/myplugin_out", "a") as output:
output.write(f"---> in 'Plugin._dispatch_request\n{name}\n"+
f"{repr(method.func)}\n"+
f"{json.dumps(result)}\n")
if not method.background:
# Only if this is a synchronous (background=False) call do we need to
# return the result. Otherwise the callee (method) will eventually need
# to call request.set_result or request.set_exception to
# return a result or raise an exception.
request.set_result(result)
except Exception as e:
if name in hook_fallbacks:
response = hook_fallbacks[name]
self.log((
"Hook handler for {name} failed with an exception. "
"Returning safe fallback response {response} to avoid "
"crashing the main daemon. Please contact the plugin "
"author!"
).format(name=name, response=response), level="error")
request.set_result(response)
else:
request.set_exception(e)
self.log(traceback.format_exc())
in order to see in the file /tmp/myplugin_out for each request from
lightningd
which function is used to produce the result field of the JSON-RPC response and
what does the result field of the JSON-RPC response contains.
Back to our terminal we restart our plugin in order to take into
account the changes we made in pyln-client package:
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.py
{
"command": "stop",
"result": "Successfully stopped myplugin.py."
}
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.py
{
"command": "start",
"plugins": [...]
}
Now we can check that the following
--------
---> in 'Plugin.run'
[b'{"jsonrpc":"2.0","id":187,"method":"getmanifest","params":{"allow-deprecated-apis":false}}', b'']
---> in 'Plugin._dispatch_request
getmanifest
<bound method Plugin._getmanifest of <pyln.client.plugin.Plugin object at 0x7f8b44d3bc10>>
{"options": [], "rpcmethods": [{"name": "node-id", "category": "plugin", "usage": "", "description": "Undocumented RPC method from a plugin."}], "subscriptions": [], "hooks": [], "dynamic": true, "nonnumericids": true, "notifications": [], "featurebits": {}}
--------
---> in 'Plugin.run'
[b'{"jsonrpc":"2.0","id":"cln:init#188","method":"init","params":{"options":{},"configuration":{"lightning-dir":"/tmp/l1-regtest/regtest","rpc-file":"lightning-rpc","startup":false,"network":"regtest","feature_set":{"init":"08a000080269a2","node":"88a000080269a2","channel":"","invoice":"02000000024100"}}}}', b'']
---> in 'Plugin._dispatch_request
init
<bound method Plugin._init of <pyln.client.plugin.Plugin object at 0x7f8b44d3bc10>>
null
has been appended to /tmp/myplugin_out file.
Let's call node-id method like this
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli node-id
{
"node_id": "03fd248f93b8352d6cfc6a168f0fd10b1462f72d84b310a56f5c18076c5cf04fe4"
}
and check that the following informations about node-id request
--------
---> in 'Plugin.run'
[b'{ "jsonrpc" : "2.0", "method" : "node-id", "id" : "cli:node-id#699655/cln:node-id#205", "params" :[ ] }', b'']
---> in 'Plugin._dispatch_request
node-id
<function node_id_func at 0x7f8b44c59fc0>
{"node_id": "03fd248f93b8352d6cfc6a168f0fd10b1462f72d84b310a56f5c18076c5cf04fe4"}
have been appended to /tmp/myplugin_out file.
Request.set_result
In Plugin._dispatch_request we built a result that we passed to
Request.set_result method which is defined in
lightning:contrib/pyln-client/pyln/client/plugin.py like this:
class Request(dict):
...
def set_result(self, result: Any) -> None:
if self.state != RequestState.PENDING:
assert(self.termination_tb is not None)
raise ValueError(
"Cannot set the result of a request that is not pending, "
"current state is {state}. Request previously terminated at\n"
"{tb}".format(state=self.state, tb=self.termination_tb))
self.result = result
self._write_result({
'jsonrpc': '2.0',
'id': self.id,
'result': self.result
})
self.state = RequestState.FINISHED
self.termination_tb = "".join(traceback.extract_stack().format()[:-1])
In that method we build the JSON-RPC response corresponding the request that Plugin.run received in the stdin stream of the plugin process. And that response is then passed to Request._write_result method
class Request(dict):
...
def _write_result(self, result: dict) -> None:
self.plugin._write_locked(result)
which delegates the work to Plugin._write_locked that finally writes
to the plugin stdout stream. This leads to reply to lightningd
request:
class Plugin(object):
...
def _write_locked(self, obj: JSONType) -> None:
# ensure_ascii turns UTF-8 into \uXXXX so we need to suppress that,
# then utf8 ourselves.
s = bytes(json.dumps(
obj,
cls=LightningRpc.LightningJSONEncoder,
ensure_ascii=False
) + "\n\n", encoding='utf-8')
with self.write_lock:
self.stdout.buffer.write(s)
self.stdout.flush()
And we are back at the beginning of Plugin.run IO loop waiting for
incoming requests from lightningd.
To see exactly what is the JSON-RPC response that we send back to
lightningd, we modify Request.set_result (in the file plugin.py in
.venv virtual environment) like this:
class Request(dict):
...
def set_result(self, result: Any) -> None:
if self.state != RequestState.PENDING:
assert(self.termination_tb is not None)
raise ValueError(
"Cannot set the result of a request that is not pending, "
"current state is {state}. Request previously terminated at\n"
"{tb}".format(state=self.state, tb=self.termination_tb))
self.result = result
response = {
'jsonrpc': '2.0',
'id': self.id,
'result': self.result
}
with open("/tmp/myplugin_out", "a") as output:
output.write(f"---> in 'Request.set_result'\n{json.dumps(response)}\n")
self._write_result(response)
self.state = RequestState.FINISHED
self.termination_tb = "".join(traceback.extract_stack().format()[:-1])
Now each time our plugin receives a request from lightningd we'll
append to /tmp/myplugin_out file the JSON-RPC response we send back to
lightningd.
Back to our terminal we restart our plugin in order to take into
account the changes we made in pyln-client package:
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.py
{
"command": "stop",
"result": "Successfully stopped myplugin.py."
}
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.py
{
"command": "start",
"plugins": [...]
}
Now we can check that the following
--------
---> in 'Plugin.run'
[b'{"jsonrpc":"2.0","id":232,"method":"getmanifest","params":{"allow-deprecated-apis":false}}', b'']
---> in 'Plugin._dispatch_request
getmanifest
<bound method Plugin._getmanifest of <pyln.client.plugin.Plugin object at 0x7fbe22f23c10>>
{"options": [], "rpcmethods": [{"name": "node-id", "category": "plugin", "usage": "", "description": "Undocumented RPC method from a plugin."}], "subscriptions": [], "hooks": [], "dynamic": true, "nonnumericids": true, "notifications": [], "featurebits": {}}
---> in 'Request.set_result'
{"jsonrpc": "2.0", "id": 232, "result": {"options": [], "rpcmethods": [{"name": "node-id", "category": "plugin", "usage": "", "description": "Undocumented RPC method from a plugin."}], "subscriptions": [], "hooks": [], "dynamic": true, "nonnumericids": true, "notifications": [], "featurebits": {}}}
--------
---> in 'Plugin.run'
[b'{"jsonrpc":"2.0","id":"cln:init#233","method":"init","params":{"options":{},"configuration":{"lightning-dir":"/tmp/l1-regtest/regtest","rpc-file":"lightning-rpc","startup":false,"network":"regtest","feature_set":{"init":"08a000080269a2","node":"88a000080269a2","channel":"","invoice":"02000000024100"}}}}', b'']
---> in 'Plugin._dispatch_request
init
<bound method Plugin._init of <pyln.client.plugin.Plugin object at 0x7fbe22f23c10>>
null
---> in 'Request.set_result'
{"jsonrpc": "2.0", "id": "cln:init#233", "result": null}
has been appended to /tmp/myplugin_out file.
Let's call node-id method like this
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli node-id
{
"node_id": "03fd248f93b8352d6cfc6a168f0fd10b1462f72d84b310a56f5c18076c5cf04fe4"
}
and check that the following informations about node-id request
--------
---> in 'Plugin.run'
[b'{ "jsonrpc" : "2.0", "method" : "node-id", "id" : "cli:node-id#700027/cln:node-id#242", "params" :[ ] }', b'']
---> in 'Plugin._dispatch_request
node-id
<function node_id_func at 0x7fbe22e41fc0>
{"node_id": "03fd248f93b8352d6cfc6a168f0fd10b1462f72d84b310a56f5c18076c5cf04fe4"}
---> in 'Request.set_result'
{"jsonrpc": "2.0", "id": "cli:node-id#700027/cln:node-id#242", "result": {"node_id": "03fd248f93b8352d6cfc6a168f0fd10b1462f72d84b310a56f5c18076c5cf04fe4"}}
have been appended to /tmp/myplugin_out file.
We are done!
Terminal session
We ran the following commands in this order:
$ python -m venv .venv
$ source .venv/bin/activate
$ pip install pyln-client
$ ./setup.sh
$ source lightning/contrib/startup_regtest.sh
$ start_ln
$ alias l1-cli
$ l1-cli plugin start $(pwd)/myplugin.py
$ l1-cli node-id
$ l1-cli plugin stop $(pwd)/myplugin.py
$ l1-cli plugin start $(pwd)/myplugin.py
$ l1-cli node-id
$ l1-cli plugin stop $(pwd)/myplugin.py
$ l1-cli plugin start $(pwd)/myplugin.py
$ l1-cli node-id
$ l1-cli plugin stop $(pwd)/myplugin.py
$ l1-cli plugin start $(pwd)/myplugin.py
$ l1-cli node-id
And below you can read the terminal session (command lines and outputs):
◉ tony@tony:~/lnroom:
$ python -m venv .venv
◉ tony@tony:~/lnroom:
$ source .venv/bin/activate
(.venv) ◉ tony@tony:~/lnroom:
$ pip install pyln-client
...
(.venv) ◉ tony@tony:~/lnroom:
$ ./setup.sh
Ubuntu 22.04.2 LTS
Python 3.10.6
lightningd v23.02.2
pyln-client 23.2
(.venv) ◉ tony@tony:~/lnroom:
$ source lightning/contrib/startup_regtest.sh
lightning-cli is /usr/local/bin/lightning-cli
lightningd is /usr/local/bin/lightningd
Useful commands:
start_ln 3: start three nodes, l1, l2, l3
connect 1 2: connect l1 and l2
fund_nodes: connect all nodes with channels, in a row
stop_ln: shutdown
destroy_ln: remove ln directories
(.venv) ◉ tony@tony:~/lnroom:
$ start_ln
Bitcoin Core starting
awaiting bitcoind...
Making "default" bitcoind wallet.
[1] 697735
[2] 697769
WARNING: eatmydata not found: instal it for faster testing
Commands:
l1-cli, l1-log,
l2-cli, l2-log,
bt-cli, stop_ln, fund_nodes
(.venv) ◉ tony@tony:~/lnroom:
$ alias l1-cli
alias l1-cli='lightning-cli --lightning-dir=/tmp/l1-regtest'
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.py
{
"command": "start",
"plugins": [
{
"name": "/usr/local/libexec/c-lightning/plugins/autoclean",
"active": true,
"dynamic": false
},
{
"name": "/usr/local/libexec/c-lightning/plugins/chanbackup",
"active": true,
"dynamic": false
},
{
"name": "/usr/local/libexec/c-lightning/plugins/bcli",
"active": true,
"dynamic": false
},
{
"name": "/usr/local/libexec/c-lightning/plugins/commando",
"active": true,
"dynamic": false
},
{
"name": "/usr/local/libexec/c-lightning/plugins/funder",
"active": true,
"dynamic": true
},
{
"name": "/usr/local/libexec/c-lightning/plugins/topology",
"active": true,
"dynamic": false
},
{
"name": "/usr/local/libexec/c-lightning/plugins/keysend",
"active": true,
"dynamic": false
},
{
"name": "/usr/local/libexec/c-lightning/plugins/offers",
"active": true,
"dynamic": true
},
{
"name": "/usr/local/libexec/c-lightning/plugins/pay",
"active": true,
"dynamic": true
},
{
"name": "/usr/local/libexec/c-lightning/plugins/txprepare",
"active": true,
"dynamic": true
},
{
"name": "/usr/local/libexec/c-lightning/plugins/spenderp",
"active": true,
"dynamic": false
},
{
"name": "/usr/local/libexec/c-lightning/plugins/sql",
"active": true,
"dynamic": true
},
{
"name": "/usr/local/libexec/c-lightning/plugins/bookkeeper",
"active": true,
"dynamic": false
},
{
"name": "/home/tony/lnroom/myplugin.py",
"active": true,
"dynamic": true
}
]
}
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli node-id
{
"node_id": "03fd248f93b8352d6cfc6a168f0fd10b1462f72d84b310a56f5c18076c5cf04fe4"
}
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.py
"command": "stop",
"result": "Successfully stopped myplugin.py."
}
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.py
{
"command": "start",
"plugins": [...]
}
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli node-id
{
"node_id": "03fd248f93b8352d6cfc6a168f0fd10b1462f72d84b310a56f5c18076c5cf04fe4"
}
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.py
{
"command": "stop",
"result": "Successfully stopped myplugin.py."
}
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.py
{
"command": "start",
"plugins": [...]
}
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli node-id
{
"node_id": "03fd248f93b8352d6cfc6a168f0fd10b1462f72d84b310a56f5c18076c5cf04fe4"
}
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.py
{
"command": "stop",
"result": "Successfully stopped myplugin.py."
}
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.py
{
"command": "start",
"plugins": [...]
}
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli node-id
{
"node_id": "03fd248f93b8352d6cfc6a168f0fd10b1462f72d84b310a56f5c18076c5cf04fe4"
}
pyln-client source code
During the demo, we've modified Request.set_result method of
pyln-client package like this:
class Request(dict):
...
def set_result(self, result: Any) -> None:
if self.state != RequestState.PENDING:
assert(self.termination_tb is not None)
raise ValueError(
"Cannot set the result of a request that is not pending, "
"current state is {state}. Request previously terminated at\n"
"{tb}".format(state=self.state, tb=self.termination_tb))
self.result = result
response = {
'jsonrpc': '2.0',
'id': self.id,
'result': self.result
}
with open("/tmp/myplugin_out", "a") as output:
output.write(f"---> in 'Request.set_result'\n{json.dumps(response)}\n")
self._write_result(response)
self.state = RequestState.FINISHED
self.termination_tb = "".join(traceback.extract_stack().format()[:-1])
We also have modified Plugin._dispatch_request and Plugin.run
methods of pyln-client package like this:
class Plugin(object):
...
def _dispatch_request(self, request: Request) -> None:
name = request.method
if name not in self.methods:
raise ValueError("No method {} found.".format(name))
method = self.methods[name]
request.background = method.background
try:
result = self._exec_func(method.func, request)
with open("/tmp/myplugin_out", "a") as output:
output.write(f"---> in 'Plugin._dispatch_request\n{name}\n"+
f"{repr(method.func)}\n"+
f"{json.dumps(result)}\n")
if not method.background:
# Only if this is a synchronous (background=False) call do we need to
# return the result. Otherwise the callee (method) will eventually need
# to call request.set_result or request.set_exception to
# return a result or raise an exception.
request.set_result(result)
except Exception as e:
if name in hook_fallbacks:
response = hook_fallbacks[name]
self.log((
"Hook handler for {name} failed with an exception. "
"Returning safe fallback response {response} to avoid "
"crashing the main daemon. Please contact the plugin "
"author!"
).format(name=name, response=response), level="error")
request.set_result(response)
else:
request.set_exception(e)
self.log(traceback.format_exc())
...
def run(self) -> None:
# If we are not running inside lightningd we'll print usage
# and some information about the plugin.
if os.environ.get('LIGHTNINGD_PLUGIN', None) != '1':
return self.print_usage()
partial = b""
for l in self.stdin.buffer:
partial += l
msgs = partial.split(b'\n\n')
if len(msgs) < 2:
continue
with open("/tmp/myplugin_out", "a") as output:
output.write(f"\n--------\n---> in 'Plugin.run'\n{repr(msgs)}\n")
partial = self._multi_dispatch(msgs)
Source code
myplugin.py
#!/usr/bin/env python
from pyln.client import Plugin
plugin = Plugin()
@plugin.method("node-id")
def node_id_func(plugin):
node_id = plugin.rpc.getinfo()["id"]
return {"node_id":node_id}
plugin.run()
myplugin_out
--------
---> in 'Plugin.run'
[b'{"jsonrpc":"2.0","id":96,"method":"getmanifest","params":{"allow-deprecated-apis":false}}', b'']
--------
---> in 'Plugin.run'
[b'{"jsonrpc":"2.0","id":"cln:init#97","method":"init","params":{"options":{},"configuration":{"lightning-dir":"/tmp/l1-regtest/regtest","rpc-file":"lightning-rpc","startup":false,"network":"regtest","feature_set":{"init":"08a000080269a2","node":"88a000080269a2","channel":"","invoice":"02000000024100"}}}}', b'']
--------
---> in 'Plugin.run'
[b'{ "jsonrpc" : "2.0", "method" : "node-id", "id" : "cli:node-id#698621/cln:node-id#108", "params" :[ ] }', b'']
--------
---> in 'Plugin.run'
[b'{"jsonrpc":"2.0","id":187,"method":"getmanifest","params":{"allow-deprecated-apis":false}}', b'']
---> in 'Plugin._dispatch_request
getmanifest
<bound method Plugin._getmanifest of <pyln.client.plugin.Plugin object at 0x7f8b44d3bc10>>
{"options": [], "rpcmethods": [{"name": "node-id", "category": "plugin", "usage": "", "description": "Undocumented RPC method from a plugin."}], "subscriptions": [], "hooks": [], "dynamic": true, "nonnumericids": true, "notifications": [], "featurebits": {}}
--------
---> in 'Plugin.run'
[b'{"jsonrpc":"2.0","id":"cln:init#188","method":"init","params":{"options":{},"configuration":{"lightning-dir":"/tmp/l1-regtest/regtest","rpc-file":"lightning-rpc","startup":false,"network":"regtest","feature_set":{"init":"08a000080269a2","node":"88a000080269a2","channel":"","invoice":"02000000024100"}}}}', b'']
---> in 'Plugin._dispatch_request
init
<bound method Plugin._init of <pyln.client.plugin.Plugin object at 0x7f8b44d3bc10>>
null
--------
---> in 'Plugin.run'
[b'{ "jsonrpc" : "2.0", "method" : "node-id", "id" : "cli:node-id#699655/cln:node-id#205", "params" :[ ] }', b'']
---> in 'Plugin._dispatch_request
node-id
<function node_id_func at 0x7f8b44c59fc0>
{"node_id": "03fd248f93b8352d6cfc6a168f0fd10b1462f72d84b310a56f5c18076c5cf04fe4"}
--------
---> in 'Plugin.run'
[b'{"jsonrpc":"2.0","id":232,"method":"getmanifest","params":{"allow-deprecated-apis":false}}', b'']
---> in 'Plugin._dispatch_request
getmanifest
<bound method Plugin._getmanifest of <pyln.client.plugin.Plugin object at 0x7fbe22f23c10>>
{"options": [], "rpcmethods": [{"name": "node-id", "category": "plugin", "usage": "", "description": "Undocumented RPC method from a plugin."}], "subscriptions": [], "hooks": [], "dynamic": true, "nonnumericids": true, "notifications": [], "featurebits": {}}
---> in 'Request.set_result'
{"jsonrpc": "2.0", "id": 232, "result": {"options": [], "rpcmethods": [{"name": "node-id", "category": "plugin", "usage": "", "description": "Undocumented RPC method from a plugin."}], "subscriptions": [], "hooks": [], "dynamic": true, "nonnumericids": true, "notifications": [], "featurebits": {}}}
--------
---> in 'Plugin.run'
[b'{"jsonrpc":"2.0","id":"cln:init#233","method":"init","params":{"options":{},"configuration":{"lightning-dir":"/tmp/l1-regtest/regtest","rpc-file":"lightning-rpc","startup":false,"network":"regtest","feature_set":{"init":"08a000080269a2","node":"88a000080269a2","channel":"","invoice":"02000000024100"}}}}', b'']
---> in 'Plugin._dispatch_request
init
<bound method Plugin._init of <pyln.client.plugin.Plugin object at 0x7fbe22f23c10>>
null
---> in 'Request.set_result'
{"jsonrpc": "2.0", "id": "cln:init#233", "result": null}
--------
---> in 'Plugin.run'
[b'{ "jsonrpc" : "2.0", "method" : "node-id", "id" : "cli:node-id#700027/cln:node-id#242", "params" :[ ] }', b'']
---> in 'Plugin._dispatch_request
node-id
<function node_id_func at 0x7fbe22e41fc0>
{"node_id": "03fd248f93b8352d6cfc6a168f0fd10b1462f72d84b310a56f5c18076c5cf04fe4"}
---> in 'Request.set_result'
{"jsonrpc": "2.0", "id": "cli:node-id#700027/cln:node-id#242", "result": {"node_id": "03fd248f93b8352d6cfc6a168f0fd10b1462f72d84b310a56f5c18076c5cf04fe4"}}
setup.sh
#!/usr/bin/env bash
ubuntu=$(lsb_release -ds)
lightningd=$(lightningd --version | xargs printf "lightningd %s\n")
python=$(python --version)
pyln_client=$(pip list | rg pyln-client)
printf "%s\n%s\n%s\n%s\n" "$ubuntu" "$python" "$lightningd" "$pyln_client"