Compare commits

...

32 commits
v0.1.5 ... main

Author SHA1 Message Date
faba2949d2 cargo: Bump hmac and sha2 dependencies.
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
Bump the `hmac` dependency to `0.12`, therefor remove deprecated
`NewMac`. Further, bump the `sha2` dependency to `0.10`.
2023-06-11 22:37:25 +02:00
57d4f10b41 cargo: Bump clap to 4.3
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
Bump `clap` to latest version.
2023-06-11 18:01:37 +02:00
35b31b2a15 cargo: Cargo update
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
Update the build dependencies by `cargo update`.
2023-06-11 17:28:01 +02:00
71153b28ec cargo: Order dependencies alphabetically
Some checks failed
ci/woodpecker/push/woodpecker Pipeline failed
To improve readability of dependencies order them alphabetically.
2023-06-11 17:28:01 +02:00
f6ec8af944 clippy: Fix clippy::get_first
Some checks failed
ci/woodpecker/push/woodpecker Pipeline failed
Fix the `clippy::get_first` lint.

Note clippy fails due to a false positive.
2023-06-11 17:27:18 +02:00
55c5134840 ci: Update CI config
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
2022-10-15 01:56:07 +02:00
f38c70373c metrics: Increase hooks_successful when done
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
2022-03-23 15:20:39 +01:00
f25ee6b943 ci: run everything in pipeline in parallel
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
2022-02-11 19:52:09 +01:00
976c25ba1a ci: Also execute the tests during ci
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
2022-02-10 01:38:37 +01:00
81be79d46d ci: Add a basic ci config
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
Do things like build docs, check fmt, clippy and build in non-release
and release mode as well as the debian package. Include a badge which
reflects the status of the ci for the main branch to the readme.
2022-02-10 00:56:49 +01:00
f195162ce5 Update build dependencies
Use clap `v3.0`.
2022-01-07 11:50:52 +01:00
0312a600ed Add from annotation to inherited errors 2022-01-07 11:49:16 +01:00
f7aea10c6b Update build dependencies 2021-12-14 17:00:55 +01:00
620fa520ce Raise version to 0.1.6 2021-12-09 13:43:46 +01:00
856cdc9457 Update build dependencies
This needed also some adaption in order to use `clap 3.0.0-rc.0`.
2021-12-09 13:40:17 +01:00
2c3319ad84 Create the webhookey manpage
Also add the manpage to the build recipe of the debian package.
This closes #13.
2021-11-26 11:59:47 +01:00
0f62ce701e Set server ident in headers
Set server ident in headers to Webhookey.
2021-11-26 11:44:59 +01:00
5f5d014bc0 Update build dependencies 2021-11-26 10:58:37 +01:00
c1b322bc52 Add link to webhooks wiki page to readme
So that we have one thing which points to an explaination about what
webhooks are.
2021-11-22 14:42:38 +01:00
506001a366 Improve error message
Write a more correct error message.
2021-11-22 14:42:10 +01:00
33e39f0b40 Improve readme and fix typos
Improve wording and fix typos in readme.
2021-11-19 17:21:29 +01:00
5775870a8e Move hook related code into hooks.rs
This concludes the code separation into several files.
2021-11-19 14:28:26 +01:00
1280352f25 Move config related code into config.rs
To continue with code separation to improve readability.
2021-11-19 13:41:48 +01:00
5e1d433c38 Move metrics related code into metrics.rs
To improve readability metrics related code moves to `metrics.rs`.
2021-11-19 11:40:21 +01:00
3a95ecfd11 Rename webook.rs to filter.rs
Also improve code separation, still more to come.
2021-11-19 11:28:37 +01:00
b7ad590d39 Create interrelate macro to evaluate filters
This reduces code duplication.
2021-11-19 11:26:53 +01:00
b8f114900b Compile regex when parsing config
The regexes are now compiled when the config is parsed and not each
time a new webhook is received.

Adapt tests to using parsed regex.
2021-11-19 11:26:23 +01:00
83785cc77d Add comments to metrics
To improve readability.
2021-11-18 00:16:09 +01:00
181edf589c Update readme with filters not and header 2021-11-17 15:19:38 +01:00
8c9d9e63f2 Add HeaderFilter for filter based on the header
This extends filtering to filter also on the received http header.
2021-11-17 15:13:12 +01:00
0d9c5f650f Update build dependencies 2021-11-17 14:07:39 +01:00
9c423b8dc8 Add Not filter to FilterType
To invert the result of filters.
2021-11-17 14:06:07 +01:00
13 changed files with 1957 additions and 1782 deletions

37
.woodpecker.yml Normal file
View file

@ -0,0 +1,37 @@
pipeline:
fmt:
group: default
image: rust_full
commands:
- cargo fmt --all -- --check
clippy:
group: default
image: rust_full
commands:
- cargo clippy --all-features
test:
group: default
image: rust_full
commands:
- cargo test
build:
group: default
image: rust_full
commands:
- cargo build
- cargo build --release
build-deb:
group: default
image: rust_full
commands:
- cargo deb
doc:
group: default
image: rust_full
commands:
- cargo doc

1335
Cargo.lock generated

File diff suppressed because it is too large Load diff

View file

@ -1,6 +1,6 @@
[package]
name = "webhookey"
version = "0.1.5"
version = "0.1.6"
authors = ["finga <webhookey@onders.org>"]
edition = "2021"
license = "GPL-3.0-or-later"
@ -11,22 +11,23 @@ description = "Trigger scripts via http(s) requests"
tls = ["rocket/tls"]
[dependencies]
anyhow = "1.0"
clap = { version = "4.3", features = ["derive"] }
dirs = "4.0"
env_logger = "0.9"
hex = "0.4"
hmac = "0.12"
ipnet = { version = "2.3", features = ["serde"] }
log = "0.4"
regex = "1.5"
rocket = "0.5.0-rc.1"
run_script = "0.9"
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
serde_regex = "1.1"
serde_yaml = "0.8"
regex = "1.5"
dirs = "4.0"
anyhow = "1.0"
log = "0.4"
env_logger = "0.9"
hmac = "0.11"
sha2 = "0.9"
hex = "0.4"
ipnet = { version = "2.3", features = ["serde"] }
sha2 = "0.10"
thiserror = "1.0"
run_script = "0.9"
clap = "3.0.0-beta.5"
[package.metadata.deb]
extended-description = "Webhookey receives requests in form of a so called Webhook as for example sent by Gitea. Those requests are matched against configured filters, if a filter matches, values from the header and the body can be passed to scripts as parameters which are then executed subsequently."
@ -36,6 +37,7 @@ assets = [
["config.yml", "etc/webhookey/", "644"],
["target/release/webhookey", "usr/bin/", "755"],
["README.md", "usr/share/doc/webhookey/README", "644"],
["webhookey.1", "usr/share/man/man1/", "644"],
["debian/service", "lib/systemd/system/webhookey.service", "644"],
]
conf-files = ["/etc/webhookey/config.yml"]

View file

@ -1,6 +1,9 @@
# Webhookey
Webhookey is a web server listening for requests as for example sent by
gitea's webhooks. Further, Webhookey allows you to specify rules
![status-badge](https://ci.onders.org/api/badges/finga/webhookey/status.svg?branch=main)
Webhookey is a web server listening for
[webhooks](https://en.wikipedia.org/wiki/Webhook) as for example sent
by Gitea's webhooks. Further, Webhookey allows you to specify rules
which are matched against the data received to trigger certain
actions.
@ -62,7 +65,10 @@ Whereas `<config_dir>` depends on the platform:
#### Metrics
A metrics page can optionally enabled to query stats of the currently
running webhookey instance. Note that stats are lost between restarts
of webhookey as those are not stored persistently.
of webhookey as those are not stored persistently. The `metrics`
structure is optional as well as the `ip_filter`. The `ip_filter`
supports either `allow` or `deny` containing a list of IPv4 and IPv6
addresses and networks.
Example:
```yaml
@ -98,16 +104,17 @@ hooks:
- secret_key_02
filter:
or:
- json:
pointer: /ref
regex: refs/heads/master
- not:
json:
pointer: /ref
regex: refs/heads/dev
- and:
- json:
pointer: /ref
regex: refs/heads/a_branch
- json:
pointer: /after
regex: f6e5fe4fe37df76629112d55cc210718b6a55e7e
- header:
field: X-Gitea-Event
regex: push
```
##### Command
@ -126,25 +133,28 @@ Use values from header fields sent with the HTTP request.
Example: `{{ header X-Gitea-Event }}`.
##### Allow and Deny
To allow or deny specific network ranges source is an optional
configuration parameter which either contains an allow or a deny field
with sequences containing networks. Note that IPv6 addresses have to
be put in single quotes due to the colons.
##### IP Filter
Specific IPv4 and IPv6 addresses and/or ranges ranges can be allowed
or denied. The `ip_filter` is optional and has to contain either an
`allow` or a `deny` field which contains a sequence of IPv4 or IPv6
addresses or CIDR network ranges. Note that IPv6 addresses have to be
quoted due to the colons.
Example:
```yaml
allow:
- 127.0.0.1
- 127.0.0.1/31
- "::1"
ip_filter:
allow:
- 127.0.0.1
- 127.0.0.1/31
- "::1"
```
```yaml
deny:
- 127.0.0.1
- 127.0.0.1/31
- "::1"
ip_filter:
deny:
- 127.0.0.1
- 127.0.0.1/31
- "::1"
```
##### Signature
@ -163,10 +173,20 @@ hook should be executed.
###### Conjunction Filters
Conjunction filters contain lists of other filters.
- `not`: Logical negation.
- `and`: Logical conjunction.
- `or`: Logical disjunction.
###### Concrete Filters
- `header`:
The `header` filter matches a regular expression on a field from the
received http(s) request header.
- `field`: The header field which should be matched.
- `regex`: Regular expression which has to match the specified
header field.
- `json`:
The `json` filter matches a regular expression on a field from the

2
Rocket.toml Normal file
View file

@ -0,0 +1,2 @@
[default]
ident = "Webhookey"

View file

@ -1,4 +1,4 @@
use clap::{crate_authors, crate_version, AppSettings, Parser};
use clap::Parser;
#[derive(Debug, Parser)]
pub enum Command {
@ -7,16 +7,10 @@ pub enum Command {
}
#[derive(Debug, Parser)]
#[clap(
version = crate_version!(),
author = crate_authors!(", "),
global_setting = AppSettings::InferSubcommands,
global_setting = AppSettings::PropagateVersion,
)]
pub struct Opts {
/// Provide a path to the configuration file
#[clap(short, long, value_name = "FILE")]
#[arg(short, long, value_name = "FILE")]
pub config: Option<String>,
#[clap(subcommand)]
#[command(subcommand)]
pub command: Option<Command>,
}

52
src/config.rs Normal file
View file

@ -0,0 +1,52 @@
use crate::{filters::IpFilter, hooks::Hook};
use anyhow::{bail, Result};
use log::info;
use serde::{Deserialize, Serialize};
use std::{collections::BTreeMap, fs::File};
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub struct MetricsConfig {
pub enabled: bool,
pub ip_filter: Option<IpFilter>,
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub struct Config {
pub metrics: Option<MetricsConfig>,
pub hooks: BTreeMap<String, Hook>,
}
pub fn get_config() -> Result<File> {
// Look for config in CWD..
if let Ok(config) = File::open("config.yml") {
info!("Loading configuration from `./config.yml`");
return Ok(config);
}
// ..look for user path config..
if let Some(mut path) = dirs::config_dir() {
path.push("webhookey/config.yml");
if let Ok(config) = File::open(&path) {
info!(
"Loading configuration from `{}`",
path.to_str().unwrap_or("<path unprintable>"),
);
return Ok(config);
}
}
// ..look for systemwide config..
if let Ok(config) = File::open("/etc/webhookey/config.yml") {
info!("Loading configuration from `/etc/webhookey/config.yml`");
return Ok(config);
}
// ..you had your chance.
bail!("No configuration file found.");
}

157
src/filters.rs Normal file
View file

@ -0,0 +1,157 @@
use crate::WebhookeyError;
use anyhow::Result;
use ipnet::IpNet;
use log::{debug, error, trace};
use regex::Regex;
use rocket::{http::HeaderMap, Request};
use serde::{Deserialize, Serialize};
use std::net::IpAddr;
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields, untagged)]
pub enum AddrType {
IpAddr(IpAddr),
IpNet(IpNet),
}
impl AddrType {
pub fn matches(&self, client_ip: &IpAddr) -> bool {
match self {
AddrType::IpAddr(addr) => addr == client_ip,
AddrType::IpNet(net) => net.contains(client_ip),
}
}
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields, rename_all = "lowercase")]
pub enum IpFilter {
Allow(Vec<AddrType>),
Deny(Vec<AddrType>),
}
impl IpFilter {
pub fn validate(&self, client_ip: &IpAddr) -> bool {
match self {
IpFilter::Allow(list) => list.iter().any(|i| i.matches(client_ip)),
IpFilter::Deny(list) => !list.iter().any(|i| i.matches(client_ip)),
}
}
}
#[derive(Debug, Deserialize, Serialize)]
pub struct HeaderFilter {
pub field: String,
#[serde(with = "serde_regex")]
pub regex: Regex,
}
impl HeaderFilter {
pub fn evaluate(&self, headers: &HeaderMap) -> Result<bool, WebhookeyError> {
trace!(
"Matching `{}` on `{}` from received header",
&self.regex,
&self.field,
);
if let Some(value) = headers.get_one(&self.field) {
if self.regex.is_match(value) {
debug!("Regex `{}` for `{}` matches", &self.regex, &self.field);
return Ok(true);
}
}
debug!(
"Regex `{}` for header field `{}` does not match",
&self.regex, &self.field
);
Ok(false)
}
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub struct JsonFilter {
pub pointer: String,
#[serde(with = "serde_regex")]
pub regex: Regex,
}
impl JsonFilter {
pub fn evaluate(&self, data: &serde_json::Value) -> Result<bool, WebhookeyError> {
trace!(
"Matching `{}` on `{}` from received json",
&self.regex,
&self.pointer,
);
if let Some(value) = data.pointer(&self.pointer) {
if self.regex.is_match(&crate::get_string(value)?) {
debug!("Regex `{}` for `{}` matches", &self.regex, &self.pointer);
return Ok(true);
}
}
debug!(
"Regex `{}` for json field `{}` does not match",
&self.regex, &self.pointer
);
Ok(false)
}
}
macro_rules! interrelate {
($request:expr, $data:expr, $filters:expr, $relation:ident) => {{
let (mut results, mut errors) = (Vec::new(), Vec::new());
$filters
.iter()
.map(|filter| filter.evaluate($request, $data))
.for_each(|item| match item {
Ok(o) => results.push(o),
Err(e) => errors.push(e),
});
if errors.is_empty() {
Ok(results.iter().$relation(|r| *r))
} else {
errors
.iter()
.for_each(|e| error!("Could not evaluate Filter: {}", e));
Err(WebhookeyError::InvalidFilter)
}
}};
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields, rename_all = "lowercase")]
pub enum FilterType {
Not(Box<FilterType>),
And(Vec<FilterType>),
Or(Vec<FilterType>),
#[serde(rename = "header")]
HeaderFilter(HeaderFilter),
#[serde(rename = "json")]
JsonFilter(JsonFilter),
}
impl FilterType {
pub fn evaluate(
&self,
request: &Request,
data: &serde_json::Value,
) -> Result<bool, WebhookeyError> {
match self {
FilterType::Not(filter) => Ok(!filter.evaluate(request, data)?),
FilterType::And(filters) => interrelate!(request, data, filters, all),
FilterType::Or(filters) => interrelate!(request, data, filters, any),
FilterType::HeaderFilter(filter) => filter.evaluate(request.headers()),
FilterType::JsonFilter(filter) => filter.evaluate(data),
}
}
}

804
src/hooks.rs Normal file
View file

@ -0,0 +1,804 @@
use crate::{
filters::{FilterType, IpFilter},
Config, Metrics, WebhookeyError,
};
use anyhow::{anyhow, bail, Result};
use hmac::{Hmac, Mac};
use log::{debug, error, info, trace, warn};
use rocket::{
data::{FromData, ToByteUnit},
futures::TryFutureExt,
http::{HeaderMap, Status},
outcome::Outcome::{self, Failure, Success},
post,
tokio::io::AsyncReadExt,
Data, Request, State,
};
use run_script::ScriptOptions;
use serde::{Deserialize, Serialize};
use sha2::Sha256;
use std::{
collections::BTreeMap,
net::{IpAddr, Ipv4Addr, SocketAddr},
sync::atomic::Ordering,
};
fn accept_ip(hook_name: &str, client_ip: &IpAddr, ip: &IpFilter) -> bool {
if ip.validate(client_ip) {
info!("Allow hook `{}` from {}", &hook_name, &client_ip);
return true;
}
warn!("Deny hook `{}` from {}", &hook_name, &client_ip);
false
}
fn get_header_field<'a>(headers: &'a HeaderMap, param: &str) -> Result<&'a str> {
headers
.get_one(param)
.ok_or_else(|| anyhow!("Could not extract event parameter from header"))
}
fn validate_request(secret: &str, signature: &str, data: &[u8]) -> Result<()> {
let mut mac = Hmac::<Sha256>::new_from_slice(secret.as_bytes())
.map_err(|e| anyhow!("Could not create hasher with secret: {}", e))?;
mac.update(data);
let raw_signature = hex::decode(signature.as_bytes())?;
mac.verify_slice(&raw_signature)
.map_err(|e| anyhow!("{}", e))
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub struct Hook {
command: String,
signature: String,
ip_filter: Option<IpFilter>,
secrets: Vec<String>,
filter: FilterType,
}
impl Hook {
fn get_command(
&self,
hook_name: &str,
request: &Request,
data: &mut serde_json::Value,
) -> Result<String> {
debug!("Replacing parameters for command of hook `{}`", hook_name);
Hook::replace_parameters(&self.command, request.headers(), data)
}
fn replace_parameters(
input: &str,
headers: &HeaderMap,
data: &serde_json::Value,
) -> Result<String> {
let mut command = String::new();
let command_template = &mut input.chars();
while let Some(i) = command_template.next() {
if i == '{' {
if let Some('{') = command_template.next() {
let mut token = String::new();
while let Some(i) = command_template.next() {
if i == '}' {
if let Some('}') = command_template.next() {
let expr = token.trim().split(' ').collect::<Vec<&str>>();
let replaced = match expr.first() {
Some(&"header") => get_header_field(
headers,
expr.get(1).ok_or_else(|| {
anyhow!("Missing parameter for `header` expression")
})?,
)?
.to_string(),
Some(pointer) => crate::get_string(
data.pointer(pointer).ok_or_else(|| {
anyhow!(
"Could not find field refered to in parameter `{}`",
pointer
)
})?,
)?,
None => bail!("Invalid expression `{}`", token),
};
command.push_str(&replaced);
trace!("Replace `{}` with: {}", token, replaced);
break;
} else {
command.push('}');
command.push(i);
}
} else {
token.push(i);
}
}
} else {
command.push('{');
command.push(i);
}
} else {
command.push(i);
}
}
Ok(command)
}
}
#[derive(Debug)]
pub struct Hooks {
pub inner: BTreeMap<String, String>,
}
impl Hooks {
pub async fn get_commands(
request: &Request<'_>,
data: Data<'_>,
) -> Result<Self, WebhookeyError> {
let mut buffer = Vec::new();
let size = data
.open(1_i32.megabytes())
.read_to_end(&mut buffer)
.map_err(WebhookeyError::Io)
.await?;
info!("Data of size {} received", size);
let config = request.guard::<&State<Config>>().await.unwrap(); // should never fail
let mut valid = false;
let mut result = BTreeMap::new();
let client_ip = &request
.client_ip()
.unwrap_or(IpAddr::V4(Ipv4Addr::UNSPECIFIED));
let hooks = config.hooks.iter().filter(|(name, hook)| {
if let Some(ip) = &hook.ip_filter {
accept_ip(name, client_ip, ip)
} else {
info!(
"Allow hook `{}` from {}, no IP filter was configured",
&name, &client_ip
);
true
}
});
for (hook_name, hook) in hooks {
let signature = request
.headers()
.get_one(&hook.signature)
.ok_or(WebhookeyError::InvalidSignature)?;
let secrets = hook
.secrets
.iter()
.map(|secret| validate_request(secret, signature, &buffer));
for secret in secrets {
match secret {
Ok(()) => {
trace!("Valid signature found for hook `{}`", hook_name);
valid = true;
let mut data: serde_json::Value =
serde_json::from_slice(&buffer).map_err(WebhookeyError::Serde)?;
match hook.filter.evaluate(request, &data) {
Ok(true) => match hook.get_command(hook_name, request, &mut data) {
Ok(command) => {
info!("Filter for `{}` matched", &hook_name);
result.insert(hook_name.to_string(), command);
break;
}
Err(e) => error!("{}", e),
},
Ok(false) => info!("Filter for `{}` did not match", &hook_name),
Err(error) => {
error!("Could not match filter for `{}`: {}", &hook_name, error)
}
}
}
Err(e) => trace!("Hook `{}` could not validate request: {}", &hook_name, e),
}
}
}
if !valid {
return Err(WebhookeyError::Unauthorized(*client_ip));
}
Ok(Hooks { inner: result })
}
}
#[rocket::async_trait]
impl<'r> FromData<'r> for Hooks {
type Error = WebhookeyError;
async fn from_data(
request: &'r Request<'_>,
data: Data<'r>,
) -> Outcome<Self, (Status, Self::Error), Data<'r>> {
{
request
.guard::<&State<Metrics>>()
.await
.unwrap() // TODO: Check if unwrap need to be fixed
.requests_received
.fetch_add(1, Ordering::Relaxed);
}
match Hooks::get_commands(request, data).await {
Ok(hooks) => {
if hooks.inner.is_empty() {
let client_ip = &request
.client_ip()
.unwrap_or(IpAddr::V4(Ipv4Addr::UNSPECIFIED));
request
.guard::<&State<Metrics>>()
.await
.unwrap() // TODO: Check if unwrap need to be fixed
.hooks_unmatched
.fetch_add(1, Ordering::Relaxed);
warn!("Unmatched hook from {}", &client_ip);
return Failure((Status::NotFound, WebhookeyError::UnmatchedHook(*client_ip)));
}
Success(hooks)
}
Err(WebhookeyError::Unauthorized(e)) => {
error!("{}", WebhookeyError::Unauthorized(e));
request
.guard::<&State<Metrics>>()
.await
.unwrap() // TODO: Check if unwrap need to be fixed
.hooks_forbidden
.fetch_add(1, Ordering::Relaxed);
Failure((Status::Unauthorized, WebhookeyError::Unauthorized(e)))
}
Err(e) => {
error!("{}", e);
request
.guard::<&State<Metrics>>()
.await
.unwrap() // TODO: Check if unwrap need to be fixed
.requests_invalid
.fetch_add(1, Ordering::Relaxed);
Failure((Status::BadRequest, e))
}
}
}
}
#[post("/", format = "json", data = "<hooks>")]
pub async fn receive_hook<'a>(
address: SocketAddr,
hooks: Hooks,
metrics: &State<Metrics>,
) -> Status {
info!("Post request received from: {}", address);
hooks.inner.iter().for_each(|(name, command)| {
info!("Execute `{}` from hook `{}`", &command, &name);
match run_script::run(command, &vec![], &ScriptOptions::new()) {
Ok((status, stdout, stderr)) => {
info!("Command `{}` exited with return code: {}", &command, status);
trace!("Output of command `{}` on stdout: {:?}", &command, &stdout);
debug!("Output of command `{}` on stderr: {:?}", &command, &stderr);
metrics.commands_executed.fetch_add(1, Ordering::Relaxed);
let _ = match status {
0 => metrics.commands_successful.fetch_add(1, Ordering::Relaxed),
_ => metrics.commands_failed.fetch_add(1, Ordering::Relaxed),
};
}
Err(e) => {
error!("Execution of `{}` failed: {}", &command, e);
metrics
.commands_execution_failed
.fetch_add(1, Ordering::Relaxed);
}
}
metrics.hooks_successful.fetch_add(1, Ordering::Relaxed);
});
Status::Ok
}
#[cfg(test)]
mod tests {
use super::*;
use crate::{
config::MetricsConfig,
filters::{AddrType, FilterType, HeaderFilter, JsonFilter},
hooks::Hook,
Metrics,
};
use regex::Regex;
use rocket::{
http::{ContentType, Header, Status},
local::asynchronous::Client,
routes,
};
use serde_json::json;
use std::collections::BTreeMap;
#[rocket::async_test]
async fn secret() {
let mut hooks = BTreeMap::new();
hooks.insert(
"test_hook".to_string(),
Hook {
command: "".to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: None,
secrets: vec!["valid".to_string()],
filter: FilterType::JsonFilter(JsonFilter {
pointer: "*".to_string(),
regex: Regex::new(".*").unwrap(),
}),
},
);
let config = Config {
metrics: None,
hooks: hooks,
};
let rocket = rocket::build()
.mount("/", routes![receive_hook])
.manage(config)
.manage(Metrics::default());
let client = Client::tracked(rocket).await.unwrap();
let response = client
.post("/")
.header(Header::new(
"X-Gitea-Signature",
"28175a0035f637f3cbb85afee9f9d319631580e7621cf790cd16ca063a2f820e",
))
.header(ContentType::JSON)
.remote("127.0.0.1:8000".parse().unwrap())
.body(&serde_json::to_string(&json!({ "foo": "bar" })).unwrap())
.dispatch();
assert_eq!(response.await.status(), Status::NotFound);
let response = client
.post("/")
.header(Header::new("X-Gitea-Signature", "beef"))
.header(ContentType::JSON)
.remote("127.0.0.1:8000".parse().unwrap())
.body(&serde_json::to_string(&json!({ "foo": "bar" })).unwrap())
.dispatch();
assert_eq!(response.await.status(), Status::Unauthorized);
let response = client
.post("/")
.header(Header::new(
"X-Gitea-Signature",
"c5c315d76318362ec129ca629b50b626bba09ad3d7ba4cc0f4c0afe4a90537a0",
))
.header(ContentType::JSON)
.remote("127.0.0.1:8000".parse().unwrap())
.body(r#"{ "not_secret": "invalid" "#)
.dispatch();
assert_eq!(response.await.status(), Status::BadRequest);
let response = client
.post("/")
.header(Header::new("X-Gitea-Signature", "foobar"))
.header(ContentType::JSON)
.remote("127.0.0.1:8000".parse().unwrap())
.dispatch();
assert_eq!(response.await.status(), Status::Unauthorized);
}
#[rocket::async_test]
async fn parse_command_request() {
let mut hooks = BTreeMap::new();
hooks.insert(
"test_hook0".to_string(),
Hook {
command:
"/usr/bin/echo {{ /repository/full_name }} --foo {{ /pull_request/base/ref }}"
.to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: None,
secrets: vec!["valid".to_string()],
filter: FilterType::JsonFilter(JsonFilter {
pointer: "/foo".to_string(),
regex: Regex::new("bar").unwrap(),
}),
},
);
hooks.insert(
"test_hook2".to_string(),
Hook {
command: "/usr/bin/echo {{ /repository/full_name }} {{ /pull_request/base/ref }}"
.to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: None,
secrets: vec!["valid".to_string()],
filter: FilterType::JsonFilter(JsonFilter {
pointer: "/foo".to_string(),
regex: Regex::new("bar").unwrap(),
}),
},
);
hooks.insert(
"test_hook3".to_string(),
Hook {
command: "/usr/bin/echo {{ /repository/full_name }} {{ /pull_request/base/ref }}"
.to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: None,
secrets: vec!["valid".to_string()],
filter: FilterType::Not(Box::new(FilterType::JsonFilter(JsonFilter {
pointer: "/foobar".to_string(),
regex: Regex::new("bar").unwrap(),
}))),
},
);
let config = Config {
metrics: None,
hooks: hooks,
};
let rocket = rocket::build()
.mount("/", routes![receive_hook])
.manage(config)
.manage(Metrics::default());
let client = Client::tracked(rocket).await.unwrap();
let response = client
.post("/")
.header(Header::new(
"X-Gitea-Signature",
"693b733871ecb684651a813c82936df683c9e4a816581f385353e06170545400",
))
.header(ContentType::JSON)
.remote("127.0.0.1:8000".parse().unwrap())
.body(
&serde_json::to_string(&json!({
"foo": "bar",
"repository": {
"full_name": "keith"
},
"pull_request": {
"base": {
"ref": "main"
}
}
}))
.unwrap(),
)
.dispatch();
assert_eq!(response.await.status(), Status::Ok);
}
#[rocket::async_test]
async fn parse_invalid_command_request() {
let mut hooks = BTreeMap::new();
hooks.insert(
"test_hook".to_string(),
Hook {
command: "/usr/bin/echo {{ /repository/full }} {{ /pull_request/base/ref }}"
.to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: None,
secrets: vec!["valid".to_string()],
filter: FilterType::JsonFilter(JsonFilter {
pointer: "/foo".to_string(),
regex: Regex::new("bar").unwrap(),
}),
},
);
let config = Config {
metrics: None,
hooks: hooks,
};
let rocket = rocket::build()
.mount("/", routes![receive_hook])
.manage(config)
.manage(Metrics::default());
let client = Client::tracked(rocket).await.unwrap();
let response = client
.post("/")
.header(Header::new(
"X-Gitea-Signature",
"693b733871ecb684651a813c82936df683c9e4a816581f385353e06170545400",
))
.header(ContentType::JSON)
.remote("127.0.0.1:8000".parse().unwrap())
.body(
&serde_json::to_string(&json!({
"foo": "bar",
"repository": {
"full_name": "keith"
},
"pull_request": {
"base": {
"ref": "main"
}
}
}))
.unwrap(),
)
.dispatch();
assert_eq!(response.await.status(), Status::NotFound);
}
#[test]
fn parse_command() {
let mut headers = HeaderMap::new();
headers.add_raw("X-Gitea-Event", "something");
assert_eq!(
Hook::replace_parameters("command", &headers, &serde_json::Value::Null).unwrap(),
"command"
);
assert_eq!(
Hook::replace_parameters(" command", &headers, &serde_json::Value::Null).unwrap(),
" command"
);
assert_eq!(
Hook::replace_parameters("command ", &headers, &serde_json::Value::Null).unwrap(),
"command "
);
assert_eq!(
Hook::replace_parameters(" command ", &headers, &serde_json::Value::Null)
.unwrap(),
" command "
);
assert_eq!(
Hook::replace_parameters("command command ", &headers, &serde_json::Value::Null)
.unwrap(),
"command command "
);
assert_eq!(
Hook::replace_parameters("{{ /foo }} command", &headers, &json!({ "foo": "bar" }))
.unwrap(),
"bar command"
);
assert_eq!(
Hook::replace_parameters(
" command {{ /foo }} ",
&headers,
&json!({ "foo": "bar" })
)
.unwrap(),
" command bar "
);
assert_eq!(
Hook::replace_parameters(
"{{ /foo }} command{{/field1/foo}}",
&headers,
&json!({ "foo": "bar", "field1": { "foo": "baz" } })
)
.unwrap(),
"bar commandbaz"
);
assert_eq!(
Hook::replace_parameters(
" command {{ /foo }} ",
&headers,
&json!({ "foo": "bar" })
)
.unwrap(),
" command bar "
);
assert_eq!(
Hook::replace_parameters(
" {{ /field1/foo }} command",
&headers,
&json!({ "field1": { "foo": "bar" } })
)
.unwrap(),
" bar command"
);
assert_eq!(
Hook::replace_parameters(
" {{ header X-Gitea-Event }} command",
&headers,
&json!({ "field1": { "foo": "bar" } })
)
.unwrap(),
" something command"
);
assert_eq!(
Hook::replace_parameters(
" {{ header X-Gitea-Event }} {{ /field1/foo }} command",
&headers,
&json!({ "field1": { "foo": "bar" } })
)
.unwrap(),
" something bar command"
);
assert_eq!(
Hook::replace_parameters(
" {{ header X-Gitea-Event }} {{ /field1/foo }} {{ /field1/bar }} {{ /field2/foo }} --command{{ /cmd }}",
&headers,
&json!({ "field1": { "foo": "bar", "bar": "baz" }, "field2": { "foo": "qux" }, "cmd": " else"})
)
.unwrap(),
" something bar baz qux --command else"
);
}
#[test]
fn parse_config() {
let config: Config = serde_yaml::from_str(
r#"---
hooks:
hook1:
command: /usr/bin/local/script_xy.sh {{ /field2/foo }} asdfasdf
signature: X-Gitea-Signature
ip_filter:
allow:
- 127.0.0.1/31
secrets:
- secret_key_01
- secret_key_02
filter:
json:
pointer: /ref
regex: refs/heads/master
hook2:
command: /usr/bin/local/script_xy.sh asdfasdf
signature: X-Gitea-Signature
secrets:
- secret_key_01
- secret_key_02
filter:
and:
- json:
pointer: /ref
regex: refs/heads/master
- header:
field: X-Gitea-Signature
regex: f6e5fe4fe37df76629112d55cc210718b6a55e7e"#,
)
.unwrap();
assert_eq!(
serde_yaml::to_string(&config).unwrap(),
serde_yaml::to_string(&Config {
metrics: None,
hooks: BTreeMap::from([
(
"hook1".to_string(),
Hook {
command: "/usr/bin/local/script_xy.sh {{ /field2/foo }} asdfasdf"
.to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: Some(IpFilter::Allow(vec![AddrType::IpNet(
"127.0.0.1/31".parse().unwrap()
)])),
secrets: vec!["secret_key_01".to_string(), "secret_key_02".to_string()],
filter: FilterType::JsonFilter(JsonFilter {
pointer: "/ref".to_string(),
regex: Regex::new("refs/heads/master").unwrap(),
}),
}
),
(
"hook2".to_string(),
Hook {
command: "/usr/bin/local/script_xy.sh asdfasdf".to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: None,
secrets: vec!["secret_key_01".to_string(), "secret_key_02".to_string()],
filter: FilterType::And(vec![
FilterType::JsonFilter(JsonFilter {
pointer: "/ref".to_string(),
regex: Regex::new("refs/heads/master").unwrap(),
}),
FilterType::HeaderFilter(HeaderFilter {
field: "X-Gitea-Signature".to_string(),
regex: Regex::new("f6e5fe4fe37df76629112d55cc210718b6a55e7e")
.unwrap(),
}),
]),
}
)
])
})
.unwrap()
);
let config: Config = serde_yaml::from_str(
r#"---
metrics:
enabled: true
hooks:
hook1:
command: /usr/bin/local/script_xy.sh {{ /field2/foo }} asdfasdf
signature: X-Gitea-Signature
ip_filter:
allow:
- 127.0.0.1/31
secrets:
- secret_key_01
- secret_key_02
filter:
json:
pointer: /ref
regex: refs/heads/master"#,
)
.unwrap();
assert_eq!(
serde_yaml::to_string(&config).unwrap(),
serde_yaml::to_string(&Config {
metrics: Some(MetricsConfig {
enabled: true,
ip_filter: None
}),
hooks: BTreeMap::from([(
"hook1".to_string(),
Hook {
command: "/usr/bin/local/script_xy.sh {{ /field2/foo }} asdfasdf"
.to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: Some(IpFilter::Allow(vec![AddrType::IpNet(
"127.0.0.1/31".parse().unwrap()
)])),
secrets: vec!["secret_key_01".to_string(), "secret_key_02".to_string()],
filter: FilterType::JsonFilter(JsonFilter {
pointer: "/ref".to_string(),
regex: Regex::new("refs/heads/master").unwrap(),
}),
}
),])
})
.unwrap()
);
}
}

View file

@ -1,454 +1,43 @@
use anyhow::{anyhow, bail, Result};
use clap::Parser;
use hmac::{Hmac, Mac, NewMac};
use log::{debug, error, info, trace, warn};
use rocket::{
data::{FromData, ToByteUnit},
futures::TryFutureExt,
get,
http::{HeaderMap, Status},
outcome::Outcome::{self, Failure, Success},
post, routes,
tokio::io::AsyncReadExt,
Data, Request, State,
};
use run_script::ScriptOptions;
use serde::{Deserialize, Serialize};
use sha2::Sha256;
use std::{
collections::BTreeMap,
fs::File,
io::BufReader,
net::{IpAddr, Ipv4Addr, SocketAddr},
sync::atomic::{AtomicUsize, Ordering},
};
mod cli;
mod webhooks;
mod config;
mod filters;
mod hooks;
mod metrics;
use crate::{
cli::Opts,
webhooks::{FilterType, IpFilter, WebhookeyError},
};
use crate::{cli::Opts, config::Config, metrics::Metrics};
use anyhow::Result;
use clap::Parser;
use log::{debug, error, trace};
use rocket::routes;
use std::{fs::File, io::BufReader, net::IpAddr};
use thiserror::Error;
#[derive(Debug, Default)]
struct WebhookeyMetrics {
requests_received: AtomicUsize,
requests_invalid: AtomicUsize,
hooks_successful: AtomicUsize,
hooks_forbidden: AtomicUsize,
hooks_unmatched: AtomicUsize,
commands_executed: AtomicUsize,
commands_execution_failed: AtomicUsize,
commands_successful: AtomicUsize,
commands_failed: AtomicUsize,
#[derive(Debug, Error)]
pub enum WebhookeyError {
#[error("Could not extract signature from header")]
InvalidSignature,
#[error("Unauthorized request from `{0}`")]
Unauthorized(IpAddr),
#[error("Unmatched hook from `{0}`")]
UnmatchedHook(IpAddr),
#[error("Could not evaluate filter request")]
InvalidFilter,
#[error("IO Error")]
Io(#[from] std::io::Error),
#[error("Serde Error")]
Serde(#[from] serde_json::Error),
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
struct MetricsConfig {
enabled: bool,
ip_filter: Option<IpFilter>,
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
struct Config {
metrics: Option<MetricsConfig>,
hooks: BTreeMap<String, Hook>,
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
struct Hook {
command: String,
signature: String,
ip_filter: Option<IpFilter>,
secrets: Vec<String>,
filter: FilterType,
}
impl Hook {
fn get_command(
&self,
hook_name: &str,
request: &Request,
data: &mut serde_json::Value,
) -> Result<String> {
debug!("Replacing parameters for command of hook `{}`", hook_name);
Hook::replace_parameters(&self.command, request.headers(), data)
}
fn replace_parameters(
input: &str,
headers: &HeaderMap,
data: &serde_json::Value,
) -> Result<String> {
let mut command = String::new();
let command_template = &mut input.chars();
while let Some(i) = command_template.next() {
if i == '{' {
if let Some('{') = command_template.next() {
let mut token = String::new();
while let Some(i) = command_template.next() {
if i == '}' {
if let Some('}') = command_template.next() {
let expr = token.trim().split(' ').collect::<Vec<&str>>();
let replaced = match expr.get(0) {
Some(&"header") => get_header_field(
headers,
expr.get(1).ok_or_else(|| {
anyhow!("Missing parameter for `header` expression")
})?,
)?
.to_string(),
Some(pointer) => webhooks::get_string(
data.pointer(pointer).ok_or_else(|| {
anyhow!(
"Could not find field refered to in parameter `{}`",
pointer
)
})?,
)?,
None => bail!("Missing expression in variable `{}`", token),
};
command.push_str(&replaced);
trace!("Replace `{}` with: {}", token, replaced);
break;
} else {
command.push('}');
command.push(i);
}
} else {
token.push(i);
}
}
} else {
command.push('{');
command.push(i);
}
} else {
command.push(i);
}
}
Ok(command)
}
}
#[derive(Debug)]
struct Hooks {
inner: BTreeMap<String, String>,
}
impl Hooks {
async fn get_commands(request: &Request<'_>, data: Data<'_>) -> Result<Self, WebhookeyError> {
let mut buffer = Vec::new();
let size = data
.open(256_i32.kilobytes())
.read_to_end(&mut buffer)
.map_err(WebhookeyError::Io)
.await?;
info!("Data of size {} received", size);
let config = request.guard::<&State<Config>>().await.unwrap(); // should never fail
let mut valid = false;
let mut result = BTreeMap::new();
let client_ip = &request
.client_ip()
.unwrap_or(IpAddr::V4(Ipv4Addr::UNSPECIFIED));
let hooks = config.hooks.iter().filter(|(name, hook)| {
if let Some(ip) = &hook.ip_filter {
accept_ip(name, client_ip, ip)
} else {
info!(
"Allow hook `{}` from {}, no IP filter was configured",
&name, &client_ip
);
true
}
});
for (hook_name, hook) in hooks {
let signature = request
.headers()
.get_one(&hook.signature)
.ok_or(WebhookeyError::InvalidSignature)?;
let secrets = hook
.secrets
.iter()
.map(|secret| validate_request(secret, signature, &buffer));
for secret in secrets {
match secret {
Ok(()) => {
trace!("Valid signature found for hook `{}`", hook_name);
valid = true;
let mut data: serde_json::Value =
serde_json::from_slice(&buffer).map_err(WebhookeyError::Serde)?;
match hook.filter.evaluate(&data) {
Ok(true) => match hook.get_command(hook_name, request, &mut data) {
Ok(command) => {
info!("Filter for `{}` matched", &hook_name);
result.insert(hook_name.to_string(), command);
break;
}
Err(e) => error!("{}", e),
},
Ok(false) => info!("Filter for `{}` did not match", &hook_name),
Err(error) => {
error!("Could not match filter for `{}`: {}", &hook_name, error)
}
}
}
Err(e) => trace!("Hook `{}` could not validate request: {}", &hook_name, e),
}
}
}
if !valid {
return Err(WebhookeyError::Unauthorized(*client_ip));
}
Ok(Hooks { inner: result })
}
}
fn accept_ip(hook_name: &str, client_ip: &IpAddr, ip: &IpFilter) -> bool {
if ip.validate(client_ip) {
info!("Allow hook `{}` from {}", &hook_name, &client_ip);
return true;
}
warn!("Deny hook `{}` from {}", &hook_name, &client_ip);
false
}
fn validate_request(secret: &str, signature: &str, data: &[u8]) -> Result<()> {
let mut mac = Hmac::<Sha256>::new_from_slice(secret.as_bytes())
.map_err(|e| anyhow!("Could not create hasher with secret: {}", e))?;
mac.update(data);
let raw_signature = hex::decode(signature.as_bytes())?;
mac.verify(&raw_signature).map_err(|e| anyhow!("{}", e))
}
fn get_header_field<'a>(headers: &'a HeaderMap, param: &str) -> Result<&'a str> {
headers
.get_one(param)
.ok_or_else(|| anyhow!("Could not extract event parameter from header"))
}
fn get_config() -> Result<File> {
// Look for config in CWD..
if let Ok(config) = File::open("config.yml") {
info!("Loading configuration from `./config.yml`");
return Ok(config);
}
// ..look for user path config..
if let Some(mut path) = dirs::config_dir() {
path.push("webhookey/config.yml");
if let Ok(config) = File::open(&path) {
info!(
"Loading configuration from `{}`",
path.to_str().unwrap_or("<path unprintable>"),
);
return Ok(config);
pub fn get_string(data: &serde_json::Value) -> Result<String, WebhookeyError> {
match &data {
serde_json::Value::Bool(bool) => Ok(bool.to_string()),
serde_json::Value::Number(number) => Ok(number.to_string()),
serde_json::Value::String(string) => Ok(string.as_str().to_string()),
x => {
error!("Could not get string from: {:?}", x);
unimplemented!()
}
}
// ..look for systemwide config..
if let Ok(config) = File::open("/etc/webhookey/config.yml") {
info!("Loading configuration from `/etc/webhookey/config.yml`");
return Ok(config);
}
// ..you had your chance.
bail!("No configuration file found.");
}
fn get_metrics(metrics: &WebhookeyMetrics) -> String {
format!(
r"# HELP webhookey_requests_received Number of requests received
# TYPE webhookey_requests_received gauge
webhookey_requests_received {}
# HELP webhookey_requests_invalid Number of invalid requests received
# TYPE webhookey_requests_invalid gauge
webhookey_requests_invalid {}
# HELP webhookey_hooks_successful Number of successfully executed hooks
# TYPE webhookey_hooks_successful gauge
webhookey_hooks_sucessful {}
# HELP webhookey_hooks_forbidden Number of forbidden requests
# TYPE webhookey_hooks_forbidden gauge
webhookey_hooks_forbidden {}
# HELP webhookey_hooks_unmatched Number of unmatched requests
# TYPE webhookey_hooks_unmatched gauge
webhookey_hooks_unmatched {}
# HELP webhookey_commands_executed Number of commands executed
# TYPE webhookey_commands_executed gauge
webhookey_commands_executed {}
# HELP webhookey_commands_execution_failed Number of commands failed to execute
# TYPE webhookey_commands_execution_failed gauge
webhookey_commands_execution_failed {}
# HELP webhookey_commands_successful Number of executed commands returning return code 0
# TYPE webhookey_commands_successful gauge
webhookey_commands_successful {}
# HELP webhookey_commands_failed Number of executed commands returning different return code than 0
# TYPE webhookey_commands_failed gauge
webhookey_commands_failed {}
",
metrics.requests_received.load(Ordering::Relaxed),
metrics.requests_invalid.load(Ordering::Relaxed),
metrics.hooks_successful.load(Ordering::Relaxed),
metrics.hooks_forbidden.load(Ordering::Relaxed),
metrics.hooks_unmatched.load(Ordering::Relaxed),
metrics.commands_executed.load(Ordering::Relaxed),
metrics.commands_execution_failed.load(Ordering::Relaxed),
metrics.commands_successful.load(Ordering::Relaxed),
metrics.commands_failed.load(Ordering::Relaxed),
)
}
#[rocket::async_trait]
impl<'r> FromData<'r> for Hooks {
type Error = WebhookeyError;
async fn from_data(
request: &'r Request<'_>,
data: Data<'r>,
) -> Outcome<Self, (Status, Self::Error), Data<'r>> {
{
request
.guard::<&State<WebhookeyMetrics>>()
.await
.unwrap() // TODO: Check if unwrap need to be fixed
.requests_received
.fetch_add(1, Ordering::Relaxed);
}
match Hooks::get_commands(request, data).await {
Ok(hooks) => {
if hooks.inner.is_empty() {
let client_ip = &request
.client_ip()
.unwrap_or(IpAddr::V4(Ipv4Addr::UNSPECIFIED));
request
.guard::<&State<WebhookeyMetrics>>()
.await
.unwrap() // TODO: Check if unwrap need to be fixed
.hooks_unmatched
.fetch_add(1, Ordering::Relaxed);
warn!("Unmatched hook from {}", &client_ip);
return Failure((Status::NotFound, WebhookeyError::UnmatchedHook(*client_ip)));
}
Success(hooks)
}
Err(WebhookeyError::Unauthorized(e)) => {
error!("{}", WebhookeyError::Unauthorized(e));
request
.guard::<&State<WebhookeyMetrics>>()
.await
.unwrap() // TODO: Check if unwrap need to be fixed
.hooks_forbidden
.fetch_add(1, Ordering::Relaxed);
Failure((Status::Unauthorized, WebhookeyError::Unauthorized(e)))
}
Err(e) => {
error!("{}", e);
request
.guard::<&State<WebhookeyMetrics>>()
.await
.unwrap() // TODO: Check if unwrap need to be fixed
.requests_invalid
.fetch_add(1, Ordering::Relaxed);
Failure((Status::BadRequest, e))
}
}
}
}
#[post("/", format = "json", data = "<hooks>")]
async fn receive_hook<'a>(
address: SocketAddr,
hooks: Hooks,
metrics: &State<WebhookeyMetrics>,
) -> Status {
info!("Post request received from: {}", address);
hooks.inner.iter().for_each(|(name, command)| {
info!("Execute `{}` from hook `{}`", &command, &name);
match run_script::run(command, &vec![], &ScriptOptions::new()) {
Ok((status, stdout, stderr)) => {
info!("Command `{}` exited with return code: {}", &command, status);
trace!("Output of command `{}` on stdout: {:?}", &command, &stdout);
debug!("Output of command `{}` on stderr: {:?}", &command, &stderr);
metrics.commands_executed.fetch_add(1, Ordering::Relaxed);
let _ = match status {
0 => metrics.commands_successful.fetch_add(1, Ordering::Relaxed),
_ => metrics.commands_failed.fetch_add(1, Ordering::Relaxed),
};
}
Err(e) => {
error!("Execution of `{}` failed: {}", &command, e);
metrics
.commands_execution_failed
.fetch_add(1, Ordering::Relaxed);
}
}
});
Status::Ok
}
#[get("/metrics")]
async fn metrics(
address: SocketAddr,
metrics: &State<WebhookeyMetrics>,
config: &State<Config>,
) -> Option<String> {
if let Some(metrics_config) = &config.metrics {
if metrics_config.enabled {
if let Some(filter) = &metrics_config.ip_filter {
if filter.validate(&address.ip()) {
return Some(get_metrics(metrics));
}
} else {
return Some(get_metrics(metrics));
}
}
}
warn!("Forbidden request for metrics: {:?}", address);
None
}
#[rocket::main]
@ -459,7 +48,7 @@ async fn main() -> Result<()> {
let config: Config = match cli.config {
Some(config) => serde_yaml::from_reader(BufReader::new(File::open(config)?))?,
_ => serde_yaml::from_reader(BufReader::new(get_config()?))?,
_ => serde_yaml::from_reader(BufReader::new(config::get_config()?))?,
};
trace!("Parsed configuration:\n{}", serde_yaml::to_string(&config)?);
@ -471,467 +60,11 @@ async fn main() -> Result<()> {
}
rocket::build()
.mount("/", routes![receive_hook, metrics])
.mount("/", routes![hooks::receive_hook, metrics::metrics])
.manage(config)
.manage(WebhookeyMetrics::default())
.manage(Metrics::default())
.launch()
.await?;
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use crate::webhooks::{AddrType, JsonFilter};
use rocket::{
http::{ContentType, Header},
local::asynchronous::Client,
};
use serde_json::json;
#[rocket::async_test]
async fn secret() {
let mut hooks = BTreeMap::new();
hooks.insert(
"test_hook".to_string(),
Hook {
command: "".to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: None,
secrets: vec!["valid".to_string()],
filter: FilterType::JsonFilter(JsonFilter {
pointer: "*".to_string(),
regex: "*".to_string(),
}),
},
);
let config = Config {
metrics: None,
hooks: hooks,
};
let rocket = rocket::build()
.mount("/", routes![receive_hook])
.manage(config)
.manage(WebhookeyMetrics::default());
let client = Client::tracked(rocket).await.unwrap();
let response = client
.post("/")
.header(Header::new(
"X-Gitea-Signature",
"28175a0035f637f3cbb85afee9f9d319631580e7621cf790cd16ca063a2f820e",
))
.header(ContentType::JSON)
.remote("127.0.0.1:8000".parse().unwrap())
.body(&serde_json::to_string(&json!({ "foo": "bar" })).unwrap())
.dispatch();
assert_eq!(response.await.status(), Status::NotFound);
let response = client
.post("/")
.header(Header::new("X-Gitea-Signature", "beef"))
.header(ContentType::JSON)
.remote("127.0.0.1:8000".parse().unwrap())
.body(&serde_json::to_string(&json!({ "foo": "bar" })).unwrap())
.dispatch();
assert_eq!(response.await.status(), Status::Unauthorized);
let response = client
.post("/")
.header(Header::new(
"X-Gitea-Signature",
"c5c315d76318362ec129ca629b50b626bba09ad3d7ba4cc0f4c0afe4a90537a0",
))
.header(ContentType::JSON)
.remote("127.0.0.1:8000".parse().unwrap())
.body(r#"{ "not_secret": "invalid" "#)
.dispatch();
assert_eq!(response.await.status(), Status::BadRequest);
let response = client
.post("/")
.header(Header::new("X-Gitea-Signature", "foobar"))
.header(ContentType::JSON)
.remote("127.0.0.1:8000".parse().unwrap())
.dispatch();
assert_eq!(response.await.status(), Status::Unauthorized);
}
#[test]
fn parse_command() {
let mut headers = HeaderMap::new();
headers.add_raw("X-Gitea-Event", "something");
assert_eq!(
Hook::replace_parameters("command", &headers, &serde_json::Value::Null).unwrap(),
"command"
);
assert_eq!(
Hook::replace_parameters(" command", &headers, &serde_json::Value::Null).unwrap(),
" command"
);
assert_eq!(
Hook::replace_parameters("command ", &headers, &serde_json::Value::Null).unwrap(),
"command "
);
assert_eq!(
Hook::replace_parameters(" command ", &headers, &serde_json::Value::Null)
.unwrap(),
" command "
);
assert_eq!(
Hook::replace_parameters("command command ", &headers, &serde_json::Value::Null)
.unwrap(),
"command command "
);
assert_eq!(
Hook::replace_parameters("{{ /foo }} command", &headers, &json!({ "foo": "bar" }))
.unwrap(),
"bar command"
);
assert_eq!(
Hook::replace_parameters(
" command {{ /foo }} ",
&headers,
&json!({ "foo": "bar" })
)
.unwrap(),
" command bar "
);
assert_eq!(
Hook::replace_parameters(
"{{ /foo }} command{{/field1/foo}}",
&headers,
&json!({ "foo": "bar", "field1": { "foo": "baz" } })
)
.unwrap(),
"bar commandbaz"
);
assert_eq!(
Hook::replace_parameters(
" command {{ /foo }} ",
&headers,
&json!({ "foo": "bar" })
)
.unwrap(),
" command bar "
);
assert_eq!(
Hook::replace_parameters(
" {{ /field1/foo }} command",
&headers,
&json!({ "field1": { "foo": "bar" } })
)
.unwrap(),
" bar command"
);
assert_eq!(
Hook::replace_parameters(
" {{ header X-Gitea-Event }} command",
&headers,
&json!({ "field1": { "foo": "bar" } })
)
.unwrap(),
" something command"
);
assert_eq!(
Hook::replace_parameters(
" {{ header X-Gitea-Event }} {{ /field1/foo }} command",
&headers,
&json!({ "field1": { "foo": "bar" } })
)
.unwrap(),
" something bar command"
);
assert_eq!(
Hook::replace_parameters(
" {{ header X-Gitea-Event }} {{ /field1/foo }} {{ /field1/bar }} {{ /field2/foo }} --command{{ /cmd }}",
&headers,
&json!({ "field1": { "foo": "bar", "bar": "baz" }, "field2": { "foo": "qux" }, "cmd": " else"})
)
.unwrap(),
" something bar baz qux --command else"
);
}
#[rocket::async_test]
async fn parse_command_request() {
let mut hooks = BTreeMap::new();
hooks.insert(
"test_hook".to_string(),
Hook {
command:
"/usr/bin/echo {{ /repository/full_name }} --foo {{ /pull_request/base/ref }}"
.to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: None,
secrets: vec!["valid".to_string()],
filter: FilterType::JsonFilter(JsonFilter {
pointer: "/foo".to_string(),
regex: "bar".to_string(),
}),
},
);
hooks.insert(
"test_hook".to_string(),
Hook {
command: "/usr/bin/echo {{ /repository/full_name }} {{ /pull_request/base/ref }}"
.to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: None,
secrets: vec!["valid".to_string()],
filter: FilterType::JsonFilter(JsonFilter {
pointer: "/foo".to_string(),
regex: "bar".to_string(),
}),
},
);
let config = Config {
metrics: None,
hooks: hooks,
};
let rocket = rocket::build()
.mount("/", routes![receive_hook])
.manage(config)
.manage(WebhookeyMetrics::default());
let client = Client::tracked(rocket).await.unwrap();
let response = client
.post("/")
.header(Header::new(
"X-Gitea-Signature",
"693b733871ecb684651a813c82936df683c9e4a816581f385353e06170545400",
))
.header(ContentType::JSON)
.remote("127.0.0.1:8000".parse().unwrap())
.body(
&serde_json::to_string(&json!({
"foo": "bar",
"repository": {
"full_name": "keith"
},
"pull_request": {
"base": {
"ref": "main"
}
}
}))
.unwrap(),
)
.dispatch();
assert_eq!(response.await.status(), Status::Ok);
}
#[rocket::async_test]
async fn parse_invalid_command_request() {
let mut hooks = BTreeMap::new();
hooks.insert(
"test_hook".to_string(),
Hook {
command: "/usr/bin/echo {{ /repository/full }} {{ /pull_request/base/ref }}"
.to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: None,
secrets: vec!["valid".to_string()],
filter: FilterType::JsonFilter(JsonFilter {
pointer: "/foo".to_string(),
regex: "bar".to_string(),
}),
},
);
let config = Config {
metrics: None,
hooks: hooks,
};
let rocket = rocket::build()
.mount("/", routes![receive_hook])
.manage(config)
.manage(WebhookeyMetrics::default());
let client = Client::tracked(rocket).await.unwrap();
let response = client
.post("/")
.header(Header::new(
"X-Gitea-Signature",
"693b733871ecb684651a813c82936df683c9e4a816581f385353e06170545400",
))
.header(ContentType::JSON)
.remote("127.0.0.1:8000".parse().unwrap())
.body(
&serde_json::to_string(&json!({
"foo": "bar",
"repository": {
"full_name": "keith"
},
"pull_request": {
"base": {
"ref": "main"
}
}
}))
.unwrap(),
)
.dispatch();
assert_eq!(response.await.status(), Status::NotFound);
}
#[test]
fn parse_config() {
let config: Config = serde_yaml::from_str(
r#"---
hooks:
hook1:
command: /usr/bin/local/script_xy.sh {{ /field2/foo }} asdfasdf
signature: X-Gitea-Signature
ip_filter:
allow:
- 127.0.0.1/31
secrets:
- secret_key_01
- secret_key_02
filter:
json:
pointer: /ref
regex: refs/heads/master
hook2:
command: /usr/bin/local/script_xy.sh asdfasdf
signature: X-Gitea-Signature
secrets:
- secret_key_01
- secret_key_02
filter:
and:
- json:
pointer: /ref
regex: refs/heads/master
- json:
pointer: /after
regex: f6e5fe4fe37df76629112d55cc210718b6a55e7e"#,
)
.unwrap();
assert_eq!(
serde_yaml::to_string(&config).unwrap(),
serde_yaml::to_string(&Config {
metrics: None,
hooks: BTreeMap::from([
(
"hook1".to_string(),
Hook {
command: "/usr/bin/local/script_xy.sh {{ /field2/foo }} asdfasdf"
.to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: Some(IpFilter::Allow(vec![AddrType::IpNet(
"127.0.0.1/31".parse().unwrap()
)])),
secrets: vec!["secret_key_01".to_string(), "secret_key_02".to_string()],
filter: FilterType::JsonFilter(JsonFilter {
pointer: "/ref".to_string(),
regex: "refs/heads/master".to_string(),
}),
}
),
(
"hook2".to_string(),
Hook {
command: "/usr/bin/local/script_xy.sh asdfasdf".to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: None,
secrets: vec!["secret_key_01".to_string(), "secret_key_02".to_string()],
filter: FilterType::And(vec![
FilterType::JsonFilter(JsonFilter {
pointer: "/ref".to_string(),
regex: "refs/heads/master".to_string(),
}),
FilterType::JsonFilter(JsonFilter {
pointer: "/after".to_string(),
regex: "f6e5fe4fe37df76629112d55cc210718b6a55e7e".to_string(),
}),
]),
}
)
])
})
.unwrap()
);
let config: Config = serde_yaml::from_str(
r#"---
metrics:
enabled: true
hooks:
hook1:
command: /usr/bin/local/script_xy.sh {{ /field2/foo }} asdfasdf
signature: X-Gitea-Signature
ip_filter:
allow:
- 127.0.0.1/31
secrets:
- secret_key_01
- secret_key_02
filter:
json:
pointer: /ref
regex: refs/heads/master"#,
)
.unwrap();
assert_eq!(
serde_yaml::to_string(&config).unwrap(),
serde_yaml::to_string(&Config {
metrics: Some(MetricsConfig {
enabled: true,
ip_filter: None
}),
hooks: BTreeMap::from([(
"hook1".to_string(),
Hook {
command: "/usr/bin/local/script_xy.sh {{ /field2/foo }} asdfasdf"
.to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: Some(IpFilter::Allow(vec![AddrType::IpNet(
"127.0.0.1/31".parse().unwrap()
)])),
secrets: vec!["secret_key_01".to_string(), "secret_key_02".to_string()],
filter: FilterType::JsonFilter(JsonFilter {
pointer: "/ref".to_string(),
regex: "refs/heads/master".to_string(),
}),
}
),])
})
.unwrap()
);
}
}

91
src/metrics.rs Normal file
View file

@ -0,0 +1,91 @@
use crate::Config;
use log::warn;
use rocket::{get, State};
use std::{
net::SocketAddr,
sync::atomic::{AtomicUsize, Ordering},
};
#[derive(Debug, Default)]
pub struct Metrics {
pub requests_received: AtomicUsize,
pub requests_invalid: AtomicUsize,
pub hooks_successful: AtomicUsize,
pub hooks_forbidden: AtomicUsize,
pub hooks_unmatched: AtomicUsize,
pub commands_executed: AtomicUsize,
pub commands_execution_failed: AtomicUsize,
pub commands_successful: AtomicUsize,
pub commands_failed: AtomicUsize,
}
#[get("/metrics")]
pub async fn metrics(
address: SocketAddr,
metrics: &State<Metrics>,
config: &State<Config>,
) -> Option<String> {
// Are metrics configured?
if let Some(metrics_config) = &config.metrics {
// Are metrics enabled?
if metrics_config.enabled {
// Is a filter configured?
if let Some(filter) = &metrics_config.ip_filter {
// Does the request match the filter?
if filter.validate(&address.ip()) {
return Some(metrics.get_metrics());
}
} else {
return Some(metrics.get_metrics());
}
}
}
warn!("Forbidden request for metrics: {:?}", address);
None
}
impl Metrics {
fn get_metrics(&self) -> String {
format!(
r"# HELP webhookey_requests_received Number of requests received
# TYPE webhookey_requests_received gauge
webhookey_requests_received {}
# HELP webhookey_requests_invalid Number of invalid requests received
# TYPE webhookey_requests_invalid gauge
webhookey_requests_invalid {}
# HELP webhookey_hooks_successful Number of successfully executed hooks
# TYPE webhookey_hooks_successful gauge
webhookey_hooks_sucessful {}
# HELP webhookey_hooks_forbidden Number of forbidden requests
# TYPE webhookey_hooks_forbidden gauge
webhookey_hooks_forbidden {}
# HELP webhookey_hooks_unmatched Number of unmatched requests
# TYPE webhookey_hooks_unmatched gauge
webhookey_hooks_unmatched {}
# HELP webhookey_commands_executed Number of commands executed
# TYPE webhookey_commands_executed gauge
webhookey_commands_executed {}
# HELP webhookey_commands_execution_failed Number of commands failed to execute
# TYPE webhookey_commands_execution_failed gauge
webhookey_commands_execution_failed {}
# HELP webhookey_commands_successful Number of executed commands returning return code 0
# TYPE webhookey_commands_successful gauge
webhookey_commands_successful {}
# HELP webhookey_commands_failed Number of executed commands returning different return code than 0
# TYPE webhookey_commands_failed gauge
webhookey_commands_failed {}
",
self.requests_received.load(Ordering::Relaxed),
self.requests_invalid.load(Ordering::Relaxed),
self.hooks_successful.load(Ordering::Relaxed),
self.hooks_forbidden.load(Ordering::Relaxed),
self.hooks_unmatched.load(Ordering::Relaxed),
self.commands_executed.load(Ordering::Relaxed),
self.commands_execution_failed.load(Ordering::Relaxed),
self.commands_successful.load(Ordering::Relaxed),
self.commands_failed.load(Ordering::Relaxed),
)
}
}

View file

@ -1,163 +0,0 @@
use anyhow::Result;
use ipnet::IpNet;
use log::{debug, error, trace};
use regex::Regex;
use serde::{Deserialize, Serialize};
use std::net::IpAddr;
use thiserror::Error;
#[derive(Debug, Error)]
pub enum WebhookeyError {
#[error("Could not extract signature from header")]
InvalidSignature,
#[error("Unauthorized request from `{0}`")]
Unauthorized(IpAddr),
#[error("Unmatched hook from `{0}`")]
UnmatchedHook(IpAddr),
#[error("Could not evaluate filter request")]
InvalidFilter,
#[error("IO Error")]
Io(std::io::Error),
#[error("Serde Error")]
Serde(serde_json::Error),
#[error("Regex Error")]
Regex(regex::Error),
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields, untagged)]
pub enum AddrType {
IpAddr(IpAddr),
IpNet(IpNet),
}
impl AddrType {
pub fn matches(&self, client_ip: &IpAddr) -> bool {
match self {
AddrType::IpAddr(addr) => addr == client_ip,
AddrType::IpNet(net) => net.contains(client_ip),
}
}
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields, rename_all = "lowercase")]
pub enum IpFilter {
Allow(Vec<AddrType>),
Deny(Vec<AddrType>),
}
impl IpFilter {
pub fn validate(&self, client_ip: &IpAddr) -> bool {
match self {
IpFilter::Allow(list) => list.iter().any(|i| i.matches(client_ip)),
IpFilter::Deny(list) => !list.iter().any(|i| i.matches(client_ip)),
}
}
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub struct JsonFilter {
pub pointer: String,
pub regex: String,
}
impl JsonFilter {
pub fn evaluate(&self, data: &serde_json::Value) -> Result<bool, WebhookeyError> {
trace!(
"Matching `{}` on `{}` from received json",
&self.regex,
&self.pointer,
);
let regex = Regex::new(&self.regex).map_err(WebhookeyError::Regex)?;
if let Some(value) = data.pointer(&self.pointer) {
if regex.is_match(&get_string(value)?) {
debug!("Regex `{}` for `{}` matches", &self.regex, &self.pointer);
return Ok(true);
}
}
debug!(
"Regex `{}` for `{}` does not match",
&self.regex, &self.pointer
);
Ok(false)
}
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields, rename_all = "lowercase")]
pub enum FilterType {
And(Vec<FilterType>),
Or(Vec<FilterType>),
#[serde(rename = "json")]
JsonFilter(JsonFilter),
}
impl FilterType {
pub fn evaluate(&self, data: &serde_json::Value) -> Result<bool, WebhookeyError> {
match self {
FilterType::And(filters) => {
let (mut results, mut errors) = (Vec::new(), Vec::new());
filters
.iter()
.map(|filter| filter.evaluate(data))
.for_each(|item| match item {
Ok(o) => results.push(o),
Err(e) => errors.push(e),
});
if errors.is_empty() {
Ok(results.iter().all(|r| *r))
} else {
errors
.iter()
.for_each(|e| error!("Could not evaluate Filter: {}", e));
Err(WebhookeyError::InvalidFilter)
}
}
FilterType::Or(filters) => {
let (mut results, mut errors) = (Vec::new(), Vec::new());
filters
.iter()
.map(|filter| filter.evaluate(data))
.for_each(|item| match item {
Ok(o) => results.push(o),
Err(e) => errors.push(e),
});
if errors.is_empty() {
Ok(results.iter().any(|r| *r))
} else {
errors
.iter()
.for_each(|e| error!("Could not evaluate Filter: {}", e));
Err(WebhookeyError::InvalidFilter)
}
}
// FilterType::HeaderFilter(filter) => todo!(),
FilterType::JsonFilter(filter) => filter.evaluate(data),
}
}
}
pub fn get_string(data: &serde_json::Value) -> Result<String, WebhookeyError> {
match &data {
serde_json::Value::Bool(bool) => Ok(bool.to_string()),
serde_json::Value::Number(number) => Ok(number.to_string()),
serde_json::Value::String(string) => Ok(string.as_str().to_string()),
x => {
error!("Could not get string from: {:?}", x);
unimplemented!()
}
}
}

57
webhookey.1 Normal file
View file

@ -0,0 +1,57 @@
.TH WEBHOOKEY 1 "26 Nov 2021" "webhookey" "Linux"
.SH NAME
webhookey \- Receive webhooks and act upon them
.SH SYNOPSIS
.B webhookey [OPTIONS] [SUBCOMMAND]
.SH DESCRIPTION
\fBwebhookey\fR receives http(s) requests in form of webhooks. Those
webhooks are matched against configured filters. If a filter matches,
a command (which can also incorporate data contained in the received
header or body) is executed.
.SH OPTIONS
.TP
.BR "-c", " --config " <FILE>
Provide a path to the configuration file.
.TP
.BR "-h", " --help"
Print help information.
.TP
.BR "-V", " --version"
Print version information.
.SH SUBCOMMAND
.TP
.B configtest
- Verifies if the configuration can be parsed without errors.
.TP
.B help
- Print the general help message or the help of the given subcommand(s).
.SH ENVIRONMENT
.TP
.B ROCKET_ADDRESS
The IP address webhookey listens on (default: 127.0.0.1).
.TP
.B ROCKET_PORT
The port webhookey listens on (default: 8000).
.TP
.B ROCKET_WORKERS
The numbers of threads to use (default: CPU core count).
.TP
.B RUST_LOG
Set the Log level, which can be one one of "error", "warn", "info",
"debug", "trace" (default: "error").
.SH EXAMPLES
.PP
webhookey configtest
.RS 4
Return either "Config is OK" and return code 0 or an error description
and return code 1.
.RE
.PP
webhookey
.RS 4
Start webhookey.
.RE
.SH REPORTING BUGS
To report any bugs file an issue at
https://git.onders.org/finga/webhookey/issues/new?template=bug.md or
send an email to <bug-report@onders.org>.