🚧 This instance is under construction; expect occasional downtime. Runners available in /repos. Questions? Ask in #wikimedia-gitlab on libera.chat, or under GitLab on Phabricator.

README.md 7.52 KB
Newer Older
1
# datasette-dashboards
Romain Clement's avatar
Romain Clement committed
2

3
> Datasette plugin providing data dashboards from metadata
4

Romain Clement's avatar
Romain Clement committed
5
[![PyPI](https://img.shields.io/pypi/v/datasette-dashboards.svg)](https://pypi.org/project/datasette-dashboards/)
6
[![Test](https://github.com/rclement/datasette-dashboards/actions/workflows/test.yml/badge.svg)](https://github.com/rclement/datasette-dashboards/actions/workflows/test.yml)
Romain Clement's avatar
Romain Clement committed
7
[![Demo](https://github.com/rclement/datasette-dashboards/actions/workflows/demo.yml/badge.svg)](https://github.com/rclement/datasette-dashboards/actions/workflows/demo.yml)
8
[![Coverage Status](https://img.shields.io/codecov/c/github/rclement/datasette-dashboards)](https://codecov.io/gh/rclement/datasette-dashboards)
Romain Clement's avatar
Romain Clement committed
9
[![License](https://img.shields.io/github/license/rclement/datasette-dashboards)](https://github.com/rclement/datasette-dashboards/blob/master/LICENSE)
Romain Clement's avatar
Romain Clement committed
10
11

Try out a live demo at [https://datasette-dashboards-demo.vercel.app](https://datasette-dashboards-demo.vercel.app/-/dashboards)
Romain Clement's avatar
Romain Clement committed
12

13
**WARNING**: this plugin is still experimental and not ready for production.
14
Some breaking changes might happen between releases before reaching a stable version.
15
16
Use it at your own risks!

Romain Clement's avatar
Romain Clement committed
17
![Datasette Dashboards Demo](https://raw.githubusercontent.com/rclement/datasette-dashboards/master/demo/datasette-dashboards-demo.png)
18

Romain Clement's avatar
Romain Clement committed
19
20
21
22
23
24
25
26
## Installation

Install this plugin in the same environment as Datasette:

```bash
$ datasette install datasette-dashboards
```

27
28
29
30
31
32
33
## Usage

Define dashboards within `metadata.yml` / `metadata.json`:

```yaml
plugins:
  datasette-dashboards:
34
    my-dashboard:
35
36
      title: My Dashboard
      description: Showing some nice metrics
Romain Clement's avatar
Romain Clement committed
37
      layout:
38
39
        - [analysis-note, events-count]
        - [analysis-note, events-source]
40
41
42
43
44
45
46
47
      filters:
        date_start:
          name: Date Start
          type: date
          default: "2021-01-01"
        date_end:
          name: Date End
          type: date
48
      charts:
49
        analysis-note:
50
51
52
53
54
          library: markdown
          display: |-
            # Analysis notes
            > A quick rundown of events statistics and KPIs

55
        events-count:
56
          title: Total number of events
Romain Clement's avatar
Romain Clement committed
57
          db: jobs
58
59
          query: SELECT count(*) as count FROM events
          library: metric
60
          display:
61
62
63
            field: count
            prefix:
            suffix:
64

65
        events-source:
66
          title: Number of events by source
Romain Clement's avatar
Romain Clement committed
67
          db: jobs
68
          query: SELECT source, count(*) as count FROM events WHERE TRUE [[ AND date >= date(:date_start) ]] [[ AND date <= date(:date_end) ]] GROUP BY source ORDER BY count DESC
69
70
71
72
73
74
          library: vega
          display:
            mark: { type: bar, tooltip: true }
            encoding:
              color: { field: source, type: nominal }
              theta: { field: count, type: quantitative }
75
76
```

Romain Clement's avatar
Romain Clement committed
77
78
A new menu entry is now available, pointing at `/-/dashboards` to access all defined dashboards.

Romain Clement's avatar
Romain Clement committed
79
### Properties
80

Romain Clement's avatar
Romain Clement committed
81
82
83
84
85
86
87
Dashboard properties:

| Property      | Type     | Description           |
| ------------- | -------- | --------------------- |
| `title`       | `string` | Dashboard title       |
| `description` | `string` | Dashboard description |
| `layout`      | `array`  | Dashboard layout      |
88
89
90
91
| `filters`     | `object` | Dashboard filters     |

Dashboard filters:

92
93
94
95
96
97
98
99
| Property  | Type               | Description                            |
| --------- | ------------------ | -------------------------------------- |
| `name`    | `string`           | Filter display name                    |
| `type`    | `string`           | Filter type (`text`, `date`, `number`) |
| `default` | `string`, `number` | (optional) Filter default value        |
| `min`     | `number`           | (optional) Filter minimum value        |
| `max`     | `number`           | (optional) Filter maximum value        |
| `step`    | `number`           | (optional) Filter stepping value       |
Romain Clement's avatar
Romain Clement committed
100
101
102
103
104
105
106
107
108
109
110

Common chart properties for all chart types:

| Property  | Type     | Description                                              |
| --------- | -------- | -------------------------------------------------------- |
| `title`   | `string` | Chart title                                              |
| `db`      | `string` | Database name against which to run the query             |
| `query`   | `string` | SQL query to run and extract data from                   |
| `library` | `string` | One of supported libraries: `vega`, `markdown`           |
| `display` | `object` | Chart display specification (depend on the used library) |

111
112
113
114
115
116
117
118
119
120
To define SQL queries using dashboard filters:

```sql
SELECT * FROM mytable [[ WHERE col >= :my_filter ]]
```

```sql
SELECT * FROM mytable WHERE TRUE [[ AND col1 = :my_filter_1 ]] [[ AND col2 = :my_filter_2 ]]
```

Romain Clement's avatar
Romain Clement committed
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
#### Vega properties

Available configuration for `vega` charts:

| Property  | Type     | Description               |
| --------- | -------- | ------------------------- |
| `library` | `string` | Must be set to `vega`     |
| `display` | `object` | Vega specification object |

Notes about the `display` property:

- Requires a valid [Vega specification object](https://vega.github.io/vega-lite/docs/)
- Some fields are pre-defined: `$schema`, `title`, `width`, `view`, `config`, `data`
- All fields are passed along as-is (overriding pre-defined fields if any)
- Only `mark` and `encoding` fields are required as the bare-minimum

#### Markdown properties

Available configuration for `markdown` chart:

| Property  | Type     | Description                                       |
| --------- | -------- | ------------------------------------------------- |
| `library` | `string` | Must be set to `markdown`                         |
| `display` | `string` | Multi-line string containing the Markdown content |

Note :

- Some common properties do not apply and can be omitted: `title`, `db`, `query`
- Markdown rendering is done by [`datasette-render-markdown`](https://datasette.io/plugins/datasette-render-markdown)
- To configure Markdown rendering, extensions can be enabled in [metadata](https://datasette.io/plugins/datasette-render-markdown#user-content-markdown-extensions)

Romain Clement's avatar
Romain Clement committed
152
153
154
155
156
157
158
159
160
161
162
163
164
165
#### Metric properties

Available configuration for `metric` chart:

| Property         | Type     | Description                               |
| ---------------- | -------- | ----------------------------------------- |
| `library`        | `string` | Must be set to `metric`                   |
| `display.field`  | `string` | Numerical field to be displayed as metric |
| `display.prefix` | `string` | Prefix to be displayed before metric      |
| `display.suffix` | `string` | Prefix to be displayed after metric       |

Note:

- The `display.field` must reference a single-numerical value from the SQL query
Romain Clement's avatar
Romain Clement committed
166
  (e.g. numerical `number` field in `SELECT count(*) as number FROM events`)
Romain Clement's avatar
Romain Clement committed
167

Romain Clement's avatar
Romain Clement committed
168
### Dashboard layout
169

Romain Clement's avatar
Romain Clement committed
170
171
The default dashboard layout will present two charts per row (one per row on mobile).
To make use of custom dashboard layout using [CSS Grid Layout](https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Grid_Layout),
172
define the `layout` array property as a grid / matrix:
Romain Clement's avatar
Romain Clement committed
173
174

- Each entry represents a row of charts
175
- Each column is referring a chart by its property name
Romain Clement's avatar
Romain Clement committed
176

Romain Clement's avatar
Romain Clement committed
177
## Development
178

Romain Clement's avatar
Romain Clement committed
179
180
To set up this plugin locally, first checkout the code.
Then create a new virtual environment and the required dependencies:
181
182
183
184
185

```bash
pipenv install -d
pipenv shell
```
Romain Clement's avatar
Romain Clement committed
186

Romain Clement's avatar
Romain Clement committed
187
188
189
190
191
192
193
194
195
To run the tests:

```bash
pytest
```

## Demo

With the developmnent environment setup, you can run the demo locally:
Romain Clement's avatar
Romain Clement committed
196
197

```bash
Romain Clement's avatar
Romain Clement committed
198
datasette --metadata demo/metadata.yml demo/jobs.db
Romain Clement's avatar
Romain Clement committed
199
```
Romain Clement's avatar
Romain Clement committed
200
201
202
203
204
205

## License

Licensed under Apache License, Version 2.0

Copyright (c) 2021 - present Romain Clement