add post: rfcartography
This commit is contained in:
error
parent
1f9a1ba2f6
commit
9422286a7a
106
content/posts/rfcartography.md
Normal file
106
content/posts/rfcartography.md
Normal file
@ -0,0 +1,106 @@
|
||||
Title: RFCartography - Visualizing relations between RFCs
|
||||
Date: 2023-02-28 19:50
|
||||
Author: Error
|
||||
Slug: rfcartography
|
||||
Summary: RFCs make up a nice set of structured data with many relations. I built a small(-ish) website that analyzes these relationships and visualizes them as graphs.
|
||||
License: CC-BY-NC
|
||||
https://creativecommons.org/licenses/by-nc/4.0/
|
||||
|
||||
## Setting Sails
|
||||
|
||||
RFCs are nice documents which explain how the internet works.
|
||||
They define protocols, describe best practices and discuss the organisation of internet infrastructure.
|
||||
Unlike some standards from other organizations, they are publicly available for free, usually written in a comprehensible manner and structured in a logical way.
|
||||
|
||||
Looking for a new project to do in my spare time at the end of 2022, I ended up thinking about how to visualize relations between RFCs.
|
||||
I always liked the way RFCs are connected to each other:
|
||||
Information about which RFC updates which, which RFC is obsoleted by which, and so on, all displayed at the top of the document, easy to find.
|
||||
So I decided to create RFCartography[°portmanteau word built out of "RFC" and "Cartography"; If you were under the impression that it has something to do with radio frequencies, you were misled, sorry.], a small tool that draws graphs of these relations.
|
||||
|
||||
|
||||
|
||||
Parsing all the RFCs in order to extract the required information would have been a lot of effort and probably prone to errors.
|
||||
Fortunately, [rfc-editor.org](https://rfc-editor.org) has a nice, machine readable RFC index file in XML format with all the meta data for each RFC. [^1]
|
||||
This includes, among other data, all the relations between RFCs:
|
||||
|
||||
obsoletes
|
||||
: points to older RFCs which are superseded by this document [^2], [^3]
|
||||
|
||||
obsoleted-by
|
||||
: points to newer RFCs which supersede this document [^2], [^3]
|
||||
|
||||
updates
|
||||
: points to older RFCs which are modified and/or extended by this document [^2], [^3]
|
||||
|
||||
updated-by
|
||||
: points to newer RFCs which modify and/or extend this document [^2], [^3]
|
||||
|
||||
is-also
|
||||
: shows which other IDs this document is known as [^2], [^3]
|
||||
|
||||
see-also
|
||||
: references other relevant documents [^2]
|
||||
|
||||
## How the Cartographer works
|
||||
|
||||
RFCartography consists out of the IndexParser component, which parses the index file and makes its information available, and the RFCartography component, which builds the requested graphs and generates SVGs from them.
|
||||
This is held together by a Flask application that initially calls the IndexParser and provides the data returned by the parser to the RFCartographer, which it then repeatedly consults to generate graphs and render SVGs.
|
||||
Further, the Flask application implements the web frontend.
|
||||
The application is made available by a web server and a wsgi server.
|
||||
|
||||
|
||||
|
||||
When a request is received, the Flask application calls the RFCartographer to generate the subgraph belonging to the requested RFC.
|
||||
In order to do this, the initial RFC is added to a queue.
|
||||
Every RFC in the queue is added to the graph and all of its references are checked and added as edges to the graph.
|
||||
If a referenced document in not in the queue and hasn't been analyzed yet, it is added to the queue.
|
||||
To prevent long waiting times, a maximum depth can be specified.
|
||||
Once the queue is empty, the subgraph generation is completed and the graph can be returned.[°I profoundly apologise for this atrociously unreadable piece of code. In my defence: it seems to be working]
|
||||
|
||||
todo: list[tuple[Document, int]] = [(core, 0)]
|
||||
done: list[Document] = []
|
||||
graph: MultiDiGraph = MultiDiGraph()
|
||||
graph.add_node(core)
|
||||
nodes[core.type].append(core)
|
||||
|
||||
while len(todo) > 0:
|
||||
node: tuple[Document, int] = todo.pop(0)
|
||||
if node[0] not in done:
|
||||
done.append(node[0])
|
||||
if node[1] < max_depth or max_depth <= 0:
|
||||
for neighbor in node[0].get_references():
|
||||
if not neighbor[1].type in node_types:
|
||||
continue
|
||||
if not graph.has_node(neighbor[1]):
|
||||
graph.add_node(neighbor[1])
|
||||
nodes[neighbor[1].type].append(neighbor[1])
|
||||
graph.add_edge(node[0], neighbor[1], reftype=neighbor[0])
|
||||
edges[neighbor[0]].append((node[0], neighbor[1]))
|
||||
todo.append((neighbor[1], node[1]+1))
|
||||
|
||||
Afterwards, the subgraph can be rendered into a SVG with the help of pyplot.
|
||||
|
||||
## Here be Dragons
|
||||
|
||||
Of course it does not work without problems.
|
||||
First of all, RFCartography is really slow.
|
||||
Generating and rendering graphs takes a while, leading to long response times.
|
||||
This can be mitigated to a degree by caching responses, but to really improve on this issue, the application should probably use a database backend.[°I'll add this to RFCartography at some point]
|
||||
|
||||
Another issue seems to be due to a bug in pyplot.
|
||||
SVGs generated by RFCartography are supposed to have hyperlinks for each node that point to a page with details about that document.
|
||||
However, this does not work in some edge cases.
|
||||
If, for example, a subgraph consists out of only one node, pyplot does not add links to it.
|
||||
Further, it does not add links to the nodes if different shapes are used for different node types.[°Using different shapes would have been nice from an accessibility point of view, but since it doesn't work at the moment, RFCartography has to rely on colors only.]
|
||||
|
||||
|
||||
|
||||
## Draw your own Maps
|
||||
|
||||
You can try RFCartography yourself on [rfcartography.net](https://rfcartography.net/ "RFCartography").
|
||||
The source code can be found [here](https://git.undefinedbehavior.de/undef/RFCartography "RFCartography - undefined git server").
|
||||
I'm not sure whether this project is actually useful for anyone, but it was fun to build it.
|
||||
|
||||
[^1]: [https://www.rfc-editor.org/rfc-index.xml](https://www.rfc-editor.org/rfc-index.xml "RFC Index")
|
||||
[^2]: [https://www.rfc-editor.org/rfc-index.xsd](https://www.rfc-editor.org/rfc-index.xsd "RFC Index Schema")
|
||||
[^3]: J. Halpern, L. Daigle and O. Kolkman. (2016, May) RFC 7841: RFC Streams, Headers, and Boilerplates. [https://www.rfc-editor.org/rfc/rfc7841](https://www.rfc-editor.org/rfc/rfc7841 "RFC7841: RFC Streams, Headers, and Boilerplates")
|
||||
94
content/posts/rfcartography/2322.svg
Normal file
94
content/posts/rfcartography/2322.svg
Normal file
@ -0,0 +1,94 @@
|
||||
<?xml version="1.0" encoding="utf-8" standalone="no"?>
|
||||
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
|
||||
<svg xmlns:xlink="http://www.w3.org/1999/xlink" width="2048px" height="2048px" viewBox="0 0 72 72" xmlns="http://www.w3.org/2000/svg" version="1.1">
|
||||
<metadata>
|
||||
<rdf:RDF xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:cc="http://creativecommons.org/ns#" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
|
||||
<cc:Work>
|
||||
<dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage"/>
|
||||
<dc:date>2023-02-28T09:29:50.176550</dc:date>
|
||||
<dc:format>image/svg+xml</dc:format>
|
||||
<dc:creator>
|
||||
<cc:Agent>
|
||||
<dc:title>Matplotlib v3.6.2, https://matplotlib.org/</dc:title>
|
||||
</cc:Agent>
|
||||
</dc:creator>
|
||||
</cc:Work>
|
||||
</rdf:RDF>
|
||||
</metadata>
|
||||
<defs>
|
||||
<style type="text/css">*{stroke-linejoin: round; stroke-linecap: butt}</style>
|
||||
</defs>
|
||||
<g id="figure_1">
|
||||
<g id="patch_1">
|
||||
<path d="M 0 72
|
||||
L 72 72
|
||||
L 72 0
|
||||
L 0 0
|
||||
z
|
||||
" style="fill: #ffffff"/>
|
||||
</g>
|
||||
<g id="axes_1">
|
||||
<g id="patch_2">
|
||||
<path d="M 2 70
|
||||
L 70 70
|
||||
L 70 2
|
||||
L 2 2
|
||||
z
|
||||
" style="fill: #ffffff"/>
|
||||
</g>
|
||||
<g id="matplotlib.axis_1">
|
||||
<g id="xtick_1"/>
|
||||
<g id="xtick_2"/>
|
||||
<g id="xtick_3"/>
|
||||
</g>
|
||||
<g id="matplotlib.axis_2">
|
||||
<g id="ytick_1"/>
|
||||
<g id="ytick_2"/>
|
||||
<g id="ytick_3"/>
|
||||
</g>
|
||||
<g id="PathCollection_1">
|
||||
<defs>
|
||||
<path id="m67d9655f41" d="M 0 5.656854
|
||||
C 1.500215 5.656854 2.939188 5.060812 4 4
|
||||
C 5.060812 2.939188 5.656854 1.500215 5.656854 0
|
||||
C 5.656854 -1.500215 5.060812 -2.939188 4 -4
|
||||
C 2.939188 -5.060812 1.500215 -5.656854 0 -5.656854
|
||||
C -1.500215 -5.656854 -2.939188 -5.060812 -4 -4
|
||||
C -5.060812 -2.939188 -5.656854 -1.500215 -5.656854 0
|
||||
C -5.656854 1.500215 -5.060812 2.939188 -4 4
|
||||
C -2.939188 5.060812 -1.500215 5.656854 0 5.656854
|
||||
z
|
||||
" style="stroke: #2072b1"/>
|
||||
</defs>
|
||||
<g clip-path="url(#p7ddbf550fc)">
|
||||
<use xlink:href="#m67d9655f41" x="36" y="36" style="fill: #2072b1; stroke: #2072b1"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="patch_3">
|
||||
<path d="M 2 70
|
||||
L 2 2
|
||||
" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="patch_4">
|
||||
<path d="M 70 70
|
||||
L 70 2
|
||||
" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="patch_5">
|
||||
<path d="M 2 70
|
||||
L 70 70
|
||||
" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="patch_6">
|
||||
<path d="M 2 2
|
||||
L 70 2
|
||||
" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
<defs>
|
||||
<clipPath id="p7ddbf550fc">
|
||||
<rect x="2" y="2" width="68" height="68"/>
|
||||
</clipPath>
|
||||
</defs>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 2.7 KiB |
35363
content/posts/rfcartography/791.svg
Normal file
35363
content/posts/rfcartography/791.svg
Normal file
File diff suppressed because it is too large
Load Diff
|
After Width: | Height: | Size: 1.6 MiB |
128
content/posts/rfcartography/components.svg
Normal file
128
content/posts/rfcartography/components.svg
Normal file
@ -0,0 +1,128 @@
|
||||
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" contentStyleType="text/css" style="width:4096px;" version="1.1" viewBox="0 0 881 201" width="881px" zoomAndPan="magnify">
|
||||
<style>
|
||||
:root {
|
||||
--background: #FFF;
|
||||
--foreground: #000;
|
||||
}
|
||||
@media (prefers-color-scheme: dark) {
|
||||
:root {
|
||||
--background: #403f3f;
|
||||
--foreground: #FFF;
|
||||
}
|
||||
}
|
||||
svg {
|
||||
background: var(--background);
|
||||
}
|
||||
g {
|
||||
fill: var(--background);
|
||||
}
|
||||
line, path {
|
||||
stroke: var(--foreground);
|
||||
}
|
||||
text {
|
||||
fill: var(--foreground);
|
||||
}
|
||||
rect {
|
||||
stroke: var(--foreground);
|
||||
}
|
||||
ellipse {
|
||||
fill: var(--background);
|
||||
stroke: var(--foreground);
|
||||
}
|
||||
</style>
|
||||
<defs/>
|
||||
<g>
|
||||
<!--cluster RFCartography-->
|
||||
<g id="cluster_RFCartography">
|
||||
<path d="M408.5,6.602 L513.5,6.602 A3.75,3.75 0 0 1 516,9.102 L523,31.6699 L716.5,31.6699 A2.5,2.5 0 0 1 719,34.1699 L719,192.102 A2.5,2.5 0 0 1 716.5,194.602 L408.5,194.602 A2.5,2.5 0 0 1 406,192.102 L406,9.102 A2.5,2.5 0 0 1 408.5,6.602 " fill="none" style="stroke-width:1.5;"/>
|
||||
<line style="stroke-width:1.5;" x1="406" x2="523" y1="31.6699" y2="31.6699"/>
|
||||
<text font-family="sans-serif" font-size="14" font-weight="bold" lengthAdjust="spacing" textLength="104" x="410" y="23.5679">RFCartography</text>
|
||||
</g>
|
||||
<!--entity Flask-->
|
||||
<g id="elem_Flask">
|
||||
<rect height="49.0679" rx="2.5" ry="2.5" style="stroke-width:0.5;" width="73" x="422" y="87.102"/>
|
||||
<rect height="10" style="stroke-width:0.5;" width="15" x="475" y="92.102"/>
|
||||
<rect height="2" style="stroke-width:0.5;" width="4" x="473" y="94.102"/>
|
||||
<rect height="2" style="stroke-width:0.5;" width="4" x="473" y="98.102"/>
|
||||
<text font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="33" x="437" y="122.0679">Flask</text>
|
||||
</g>
|
||||
<!--entity RFCartographer-->
|
||||
<g id="elem_RFCartographer">
|
||||
<rect height="49.0679" rx="2.5" ry="2.5" style="stroke-width:0.5;" width="147" x="556" y="129.102"/>
|
||||
<rect height="10" style="stroke-width:0.5;" width="15" x="683" y="134.102"/>
|
||||
<rect height="2" style="stroke-width:0.5;" width="4" x="681" y="136.102"/>
|
||||
<rect height="2" style="stroke-width:0.5;" width="4" x="681" y="140.102"/>
|
||||
<text font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="107" x="571" y="164.0679">RFCartographer</text>
|
||||
</g>
|
||||
<!--entity IndexParser-->
|
||||
<g id="elem_IndexParser">
|
||||
<rect height="49.0679" rx="2.5" ry="2.5" style="stroke-width:0.5;" width="121" x="569" y="45.102"/>
|
||||
<rect height="10" style="stroke-width:0.5;" width="15" x="670" y="50.102"/>
|
||||
<rect height="2" style="stroke-width:0.5;" width="4" x="668" y="52.102"/>
|
||||
<rect height="2" style="stroke-width:0.5;" width="4" x="668" y="56.102"/>
|
||||
<text font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="81" x="584" y="80.0679">IndexParser</text>
|
||||
</g>
|
||||
<!--entity user-->
|
||||
<g id="elem_user">
|
||||
<ellipse cx="21" cy="80.602" rx="8" ry="8" style="stroke:var(--foreground);fill:var(--background);stroke-width:0.5;"/>
|
||||
<path d="M21,88.602 L21,115.602 M8,96.602 L34,96.602 M21,115.602 L8,130.602 M21,115.602 L34,130.602 " fill="none" style="stroke-width:0.5;"/>
|
||||
<text font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="30" x="6" y="147.0679">user</text>
|
||||
</g>
|
||||
<!--entity nginx-->
|
||||
<g id="elem_nginx">
|
||||
<rect height="39.0679" rx="2.5" ry="2.5" style="stroke-width:0.5;" width="58" x="138" y="92.102"/>
|
||||
<text font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="38" x="148" y="117.0679">nginx</text>
|
||||
</g>
|
||||
<!--entity uwsgi-->
|
||||
<g id="elem_uwsgi">
|
||||
<rect height="39.0679" rx="2.5" ry="2.5" style="stroke-width:0.5;" width="60" x="299" y="92.102"/>
|
||||
<text font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="40" x="309" y="117.0679">uwsgi</text>
|
||||
</g>
|
||||
<!--entity rfc-index.xml-->
|
||||
<g id="elem_rfc-index.xml">
|
||||
<path d="M766,52.602 L766,86.6699 A2.5,2.5 0 0 0 768.5,89.1699 L871.5,89.1699 A2.5,2.5 0 0 0 874,86.6699 L874,60.102 L864,50.102 L768.5,50.102 A2.5,2.5 0 0 0 766,52.602 " style="stroke-width:0.5;"/>
|
||||
<path d="M864,50.102 L864,57.602 A2.5,2.5 0 0 0 866.5,60.102 L874,60.102 " style="stroke-width:0.5;"/>
|
||||
<text font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="88" x="776" y="75.0679">rfc-index.xml</text>
|
||||
</g>
|
||||
<!--link user to nginx-->
|
||||
<g id="link_user_nginx">
|
||||
<path d="M36.33,111.602 C59.88,111.602 107.18,111.602 137.67,111.602 " fill="none" id="user-nginx" style="stroke-width:1.0;"/>
|
||||
<text font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="26" x="74" y="100.499">http</text>
|
||||
<path d="M77.3264,104.5309 A10,10 0 0 0 77.3264 118.673" fill="none" style="stroke-width:1.5;"/>
|
||||
<ellipse cx="84.3975" cy="111.602" rx="6" ry="6" style="stroke-width:1.5;"/>
|
||||
</g>
|
||||
<!--link nginx to uwsgi-->
|
||||
<g id="link_nginx_uwsgi">
|
||||
<path d="M196.24,111.602 C225.03,111.602 269.51,111.602 298.69,111.602 " fill="none" id="nginx-uwsgi" style="stroke-width:1.0;"/>
|
||||
<text font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="27" x="234" y="100.499">wsgi</text>
|
||||
<path d="M240.2477,104.5309 A10,10 0 0 0 240.2477 118.673" fill="none" style="stroke-width:1.5;"/>
|
||||
<ellipse cx="247.3188" cy="111.602" rx="6" ry="6" style="stroke-width:1.5;"/>
|
||||
</g>
|
||||
<!--link uwsgi to Flask-->
|
||||
<g id="link_uwsgi_Flask">
|
||||
<path d="M359.25,111.602 C377.78,111.602 401.87,111.602 421.56,111.602 " fill="none" id="uwsgi-Flask" style="stroke-width:1.0;"/>
|
||||
<path d="M382.8989,104.5309 A10,10 0 0 0 382.8989 118.673" fill="none" style="stroke-width:1.5;"/>
|
||||
<ellipse cx="389.97" cy="111.602" rx="6" ry="6" style="stroke-width:1.5;"/>
|
||||
</g>
|
||||
<!--link Flask to RFCartographer-->
|
||||
<g id="link_Flask_RFCartographer">
|
||||
<path d="M495.34,120.512 C512.97,124.892 534.9,130.342 555.83,135.542 " fill="none" id="Flask-RFCartographer" style="stroke-width:1.0;"/>
|
||||
<path d="M519.1903,119.1519 A10,10 0 0 0 515.7799 132.8767" fill="none" style="stroke-width:1.5;"/>
|
||||
<ellipse cx="524.3475" cy="127.7195" rx="6" ry="6" style="stroke-width:1.5;"/>
|
||||
</g>
|
||||
<!--link Flask to IndexParser-->
|
||||
<g id="link_Flask_IndexParser">
|
||||
<path d="M495.34,102.692 C516.66,97.402 544.24,90.542 568.77,84.442 " fill="none" id="Flask-IndexParser" style="stroke-width:1.0;"/>
|
||||
<path d="M522.2832,88.7144 A10,10 0 0 0 525.6949 102.4388" fill="none" style="stroke-width:1.5;"/>
|
||||
<ellipse cx="530.8512" cy="93.8707" rx="6" ry="6" style="stroke-width:1.5;"/>
|
||||
</g>
|
||||
<!--link IndexParser to rfc-index.xml-->
|
||||
<g id="link_IndexParser_rfc-index.xml">
|
||||
<path d="M690.4,69.602 C714.55,69.602 742.18,69.602 765.63,69.602 " fill="none" id="IndexParser-rfc-index.xml" style="stroke-width:1.0;"/>
|
||||
<path d="M721.2064,62.5309 A10,10 0 0 0 721.2064 76.673" fill="none" style="stroke-width:1.5;"/>
|
||||
<ellipse cx="728.2775" cy="69.602" rx="6" ry="6" style="stroke-width:1.5;"/>
|
||||
</g>
|
||||
<!--SRC=[NOwx2iCm34LtVON8r2MbSzT2e5lf7nZ7SKmSsoWAIIdzz_hHmSrYTCyLyh5gO6IFPFJ8s1jIaert04Ao9rmNHI2qSenE6xV_vdLRj84Wv1GpDi_k9eYz9X76LuDlG9H-3jwbtSwAPVeZzNeyNrTtrwlz8IXz-u71s7YT1jcQnC_xR1k0v8JjwT8M9RpYo1oWXCopNkJPieJCYqf-2pYOL4qjKisbF2Xz0G00]-->
|
||||
</g>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 8.0 KiB |
Loading…
Reference in New Issue
Block a user