finga/webhookey
1
0
Fork 0
Code Issues 5 Pull requests Projects Releases Wiki Activity

Compare commits

base: finga:main
Branches Tags
finga:main
finga:testing
finga:dev/auth
finga:dev/hermit
finga:v0.1.6
finga:v0.1.5
finga:v0.1.4
finga:v0.1.3
finga:v0.1.2
finga:v0.1.1
finga:v0.1.0
finga:v0.1.0-rc.2
finga:v0.1.0-rc.1
finga:v0.1.0-rc.0
..
compare: finga:v0.1.5
Branches Tags
finga:testing
finga:main
finga:dev/auth
finga:dev/hermit
finga:v0.1.6
finga:v0.1.5
finga:v0.1.4
finga:v0.1.3
finga:v0.1.2
finga:v0.1.1
finga:v0.1.0
finga:v0.1.0-rc.2
finga:v0.1.0-rc.1
finga:v0.1.0-rc.0

No commits in common. "main" and "v0.1.5" have entirely different histories.
main ... v0.1.5

13 changed files with 1774 additions and 1949 deletions
Download patch file Download diff file Expand all files Collapse all files

37
.woodpecker.yml
View file

@ -1,37 +0,0 @@
pipeline:
fmt:
group: default
image: rust_full
commands:
- cargo fmt --all -- --check
clippy:
group: default
image: rust_full
commands:
- cargo clippy --all-features
test:
group: default
image: rust_full
commands:
- cargo test
build:
group: default
image: rust_full
commands:
- cargo build
- cargo build --release
build-deb:
group: default
image: rust_full
commands:
- cargo deb
doc:
group: default
image: rust_full
commands:
- cargo doc

1317
Cargo.lock generated
View file

File diff suppressed because it is too large Load diff

26
Cargo.toml
View file

@ -1,6 +1,6 @@
[package]
name = "webhookey"
version = "0.1.6"
version = "0.1.5"
authors = ["finga <webhookey@onders.org>"]
edition = "2021"
license = "GPL-3.0-or-later"
@ -11,23 +11,22 @@ description = "Trigger scripts via http(s) requests"
tls = ["rocket/tls"]
[dependencies]
anyhow = "1.0"
clap = { version = "4.3", features = ["derive"] }
dirs = "4.0"
env_logger = "0.9"
hex = "0.4"
hmac = "0.12"
ipnet = { version = "2.3", features = ["serde"] }
log = "0.4"
regex = "1.5"
rocket = "0.5.0-rc.1"
run_script = "0.9"
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
serde_regex = "1.1"
serde_yaml = "0.8"
sha2 = "0.10"
regex = "1.5"
dirs = "4.0"
anyhow = "1.0"
log = "0.4"
env_logger = "0.9"
hmac = "0.11"
sha2 = "0.9"
hex = "0.4"
ipnet = { version = "2.3", features = ["serde"] }
thiserror = "1.0"
run_script = "0.9"
clap = "3.0.0-beta.5"
[package.metadata.deb]
extended-description = "Webhookey receives requests in form of a so called Webhook as for example sent by Gitea. Those requests are matched against configured filters, if a filter matches, values from the header and the body can be passed to scripts as parameters which are then executed subsequently."
@ -37,7 +36,6 @@ assets = [
["config.yml", "etc/webhookey/", "644"],
["target/release/webhookey", "usr/bin/", "755"],
["README.md", "usr/share/doc/webhookey/README", "644"],
["webhookey.1", "usr/share/man/man1/", "644"],
["debian/service", "lib/systemd/system/webhookey.service", "644"],
]
conf-files = ["/etc/webhookey/config.yml"]

64
README.md
View file

@ -1,9 +1,6 @@
# Webhookey
![status-badge](https://ci.onders.org/api/badges/finga/webhookey/status.svg?branch=main)
Webhookey is a web server listening for
[webhooks](https://en.wikipedia.org/wiki/Webhook) as for example sent
by Gitea's webhooks. Further, Webhookey allows you to specify rules
Webhookey is a web server listening for requests as for example sent by
gitea's webhooks. Further, Webhookey allows you to specify rules
which are matched against the data received to trigger certain
actions.
@ -65,10 +62,7 @@ Whereas `<config_dir>` depends on the platform:
#### Metrics
A metrics page can optionally enabled to query stats of the currently
running webhookey instance. Note that stats are lost between restarts
of webhookey as those are not stored persistently. The `metrics`
structure is optional as well as the `ip_filter`. The `ip_filter`
supports either `allow` or `deny` containing a list of IPv4 and IPv6
addresses and networks.
of webhookey as those are not stored persistently.
Example:
```yaml
@ -104,17 +98,16 @@ hooks:
- secret_key_02
filter:
or:
- not:
json:
pointer: /ref
regex: refs/heads/dev
- json:
pointer: /ref
regex: refs/heads/master
- and:
- json:
pointer: /ref
regex: refs/heads/a_branch
- header:
field: X-Gitea-Event
regex: push
- json:
pointer: /after
regex: f6e5fe4fe37df76629112d55cc210718b6a55e7e
```
##### Command
@ -133,28 +126,25 @@ Use values from header fields sent with the HTTP request.
Example: `{{ header X-Gitea-Event }}`.
##### IP Filter
Specific IPv4 and IPv6 addresses and/or ranges ranges can be allowed
or denied. The `ip_filter` is optional and has to contain either an
`allow` or a `deny` field which contains a sequence of IPv4 or IPv6
addresses or CIDR network ranges. Note that IPv6 addresses have to be
quoted due to the colons.
##### Allow and Deny
To allow or deny specific network ranges source is an optional
configuration parameter which either contains an allow or a deny field
with sequences containing networks. Note that IPv6 addresses have to
be put in single quotes due to the colons.
Example:
```yaml
ip_filter:
allow:
- 127.0.0.1
- 127.0.0.1/31
- "::1"
allow:
- 127.0.0.1
- 127.0.0.1/31
- "::1"
```
```yaml
ip_filter:
deny:
- 127.0.0.1
- 127.0.0.1/31
- "::1"
deny:
- 127.0.0.1
- 127.0.0.1/31
- "::1"
```
##### Signature
@ -173,20 +163,10 @@ hook should be executed.
###### Conjunction Filters
Conjunction filters contain lists of other filters.
- `not`: Logical negation.
- `and`: Logical conjunction.
- `or`: Logical disjunction.
###### Concrete Filters
- `header`:
The `header` filter matches a regular expression on a field from the
received http(s) request header.
- `field`: The header field which should be matched.
- `regex`: Regular expression which has to match the specified
header field.
- `json`:
The `json` filter matches a regular expression on a field from the

2
Rocket.toml
View file

@ -1,2 +0,0 @@
[default]
ident = "Webhookey"

12
src/cli.rs
View file

@ -1,4 +1,4 @@
use clap::Parser;
use clap::{crate_authors, crate_version, AppSettings, Parser};
#[derive(Debug, Parser)]
pub enum Command {
@ -7,10 +7,16 @@ pub enum Command {
}
#[derive(Debug, Parser)]
#[clap(
version = crate_version!(),
author = crate_authors!(", "),
global_setting = AppSettings::InferSubcommands,
global_setting = AppSettings::PropagateVersion,
)]
pub struct Opts {
/// Provide a path to the configuration file
#[arg(short, long, value_name = "FILE")]
#[clap(short, long, value_name = "FILE")]
pub config: Option<String>,
#[command(subcommand)]
#[clap(subcommand)]
pub command: Option<Command>,
}

52
src/config.rs
View file

@ -1,52 +0,0 @@
use crate::{filters::IpFilter, hooks::Hook};
use anyhow::{bail, Result};
use log::info;
use serde::{Deserialize, Serialize};
use std::{collections::BTreeMap, fs::File};
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub struct MetricsConfig {
pub enabled: bool,
pub ip_filter: Option<IpFilter>,
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub struct Config {
pub metrics: Option<MetricsConfig>,
pub hooks: BTreeMap<String, Hook>,
}
pub fn get_config() -> Result<File> {
// Look for config in CWD..
if let Ok(config) = File::open("config.yml") {
info!("Loading configuration from `./config.yml`");
return Ok(config);
}
// ..look for user path config..
if let Some(mut path) = dirs::config_dir() {
path.push("webhookey/config.yml");
if let Ok(config) = File::open(&path) {
info!(
"Loading configuration from `{}`",
path.to_str().unwrap_or("<path unprintable>"),
);
return Ok(config);
}
}
// ..look for systemwide config..
if let Ok(config) = File::open("/etc/webhookey/config.yml") {
info!("Loading configuration from `/etc/webhookey/config.yml`");
return Ok(config);
}
// ..you had your chance.
bail!("No configuration file found.");
}

157
src/filters.rs
View file

@ -1,157 +0,0 @@
use crate::WebhookeyError;
use anyhow::Result;
use ipnet::IpNet;
use log::{debug, error, trace};
use regex::Regex;
use rocket::{http::HeaderMap, Request};
use serde::{Deserialize, Serialize};
use std::net::IpAddr;
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields, untagged)]
pub enum AddrType {
IpAddr(IpAddr),
IpNet(IpNet),
}
impl AddrType {
pub fn matches(&self, client_ip: &IpAddr) -> bool {
match self {
AddrType::IpAddr(addr) => addr == client_ip,
AddrType::IpNet(net) => net.contains(client_ip),
}
}
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields, rename_all = "lowercase")]
pub enum IpFilter {
Allow(Vec<AddrType>),
Deny(Vec<AddrType>),
}
impl IpFilter {
pub fn validate(&self, client_ip: &IpAddr) -> bool {
match self {
IpFilter::Allow(list) => list.iter().any(|i| i.matches(client_ip)),
IpFilter::Deny(list) => !list.iter().any(|i| i.matches(client_ip)),
}
}
}
#[derive(Debug, Deserialize, Serialize)]
pub struct HeaderFilter {
pub field: String,
#[serde(with = "serde_regex")]
pub regex: Regex,
}
impl HeaderFilter {
pub fn evaluate(&self, headers: &HeaderMap) -> Result<bool, WebhookeyError> {
trace!(
"Matching `{}` on `{}` from received header",
&self.regex,
&self.field,
);
if let Some(value) = headers.get_one(&self.field) {
if self.regex.is_match(value) {
debug!("Regex `{}` for `{}` matches", &self.regex, &self.field);
return Ok(true);
}
}
debug!(
"Regex `{}` for header field `{}` does not match",
&self.regex, &self.field
);
Ok(false)
}
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub struct JsonFilter {
pub pointer: String,
#[serde(with = "serde_regex")]
pub regex: Regex,
}
impl JsonFilter {
pub fn evaluate(&self, data: &serde_json::Value) -> Result<bool, WebhookeyError> {
trace!(
"Matching `{}` on `{}` from received json",
&self.regex,
&self.pointer,
);
if let Some(value) = data.pointer(&self.pointer) {
if self.regex.is_match(&crate::get_string(value)?) {
debug!("Regex `{}` for `{}` matches", &self.regex, &self.pointer);
return Ok(true);
}
}
debug!(
"Regex `{}` for json field `{}` does not match",
&self.regex, &self.pointer
);
Ok(false)
}
}
macro_rules! interrelate {
($request:expr, $data:expr, $filters:expr, $relation:ident) => {{
let (mut results, mut errors) = (Vec::new(), Vec::new());
$filters
.iter()
.map(|filter| filter.evaluate($request, $data))
.for_each(|item| match item {
Ok(o) => results.push(o),
Err(e) => errors.push(e),
});
if errors.is_empty() {
Ok(results.iter().$relation(|r| *r))
} else {
errors
.iter()
.for_each(|e| error!("Could not evaluate Filter: {}", e));
Err(WebhookeyError::InvalidFilter)
}
}};
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields, rename_all = "lowercase")]
pub enum FilterType {
Not(Box<FilterType>),
And(Vec<FilterType>),
Or(Vec<FilterType>),
#[serde(rename = "header")]
HeaderFilter(HeaderFilter),
#[serde(rename = "json")]
JsonFilter(JsonFilter),
}
impl FilterType {
pub fn evaluate(
&self,
request: &Request,
data: &serde_json::Value,
) -> Result<bool, WebhookeyError> {
match self {
FilterType::Not(filter) => Ok(!filter.evaluate(request, data)?),
FilterType::And(filters) => interrelate!(request, data, filters, all),
FilterType::Or(filters) => interrelate!(request, data, filters, any),
FilterType::HeaderFilter(filter) => filter.evaluate(request.headers()),
FilterType::JsonFilter(filter) => filter.evaluate(data),
}
}
}

804
src/hooks.rs
View file

@ -1,804 +0,0 @@
use crate::{
filters::{FilterType, IpFilter},
Config, Metrics, WebhookeyError,
};
use anyhow::{anyhow, bail, Result};
use hmac::{Hmac, Mac};
use log::{debug, error, info, trace, warn};
use rocket::{
data::{FromData, ToByteUnit},
futures::TryFutureExt,
http::{HeaderMap, Status},
outcome::Outcome::{self, Failure, Success},
post,
tokio::io::AsyncReadExt,
Data, Request, State,
};
use run_script::ScriptOptions;
use serde::{Deserialize, Serialize};
use sha2::Sha256;
use std::{
collections::BTreeMap,
net::{IpAddr, Ipv4Addr, SocketAddr},
sync::atomic::Ordering,
};
fn accept_ip(hook_name: &str, client_ip: &IpAddr, ip: &IpFilter) -> bool {
if ip.validate(client_ip) {
info!("Allow hook `{}` from {}", &hook_name, &client_ip);
return true;
}
warn!("Deny hook `{}` from {}", &hook_name, &client_ip);
false
}
fn get_header_field<'a>(headers: &'a HeaderMap, param: &str) -> Result<&'a str> {
headers
.get_one(param)
.ok_or_else(|| anyhow!("Could not extract event parameter from header"))
}
fn validate_request(secret: &str, signature: &str, data: &[u8]) -> Result<()> {
let mut mac = Hmac::<Sha256>::new_from_slice(secret.as_bytes())
.map_err(|e| anyhow!("Could not create hasher with secret: {}", e))?;
mac.update(data);
let raw_signature = hex::decode(signature.as_bytes())?;
mac.verify_slice(&raw_signature)
.map_err(|e| anyhow!("{}", e))
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub struct Hook {
command: String,
signature: String,
ip_filter: Option<IpFilter>,
secrets: Vec<String>,
filter: FilterType,
}
impl Hook {
fn get_command(
&self,
hook_name: &str,
request: &Request,
data: &mut serde_json::Value,
) -> Result<String> {
debug!("Replacing parameters for command of hook `{}`", hook_name);
Hook::replace_parameters(&self.command, request.headers(), data)
}
fn replace_parameters(
input: &str,
headers: &HeaderMap,
data: &serde_json::Value,
) -> Result<String> {
let mut command = String::new();
let command_template = &mut input.chars();
while let Some(i) = command_template.next() {
if i == '{' {
if let Some('{') = command_template.next() {
let mut token = String::new();
while let Some(i) = command_template.next() {
if i == '}' {
if let Some('}') = command_template.next() {
let expr = token.trim().split(' ').collect::<Vec<&str>>();
let replaced = match expr.first() {
Some(&"header") => get_header_field(
headers,
expr.get(1).ok_or_else(|| {
anyhow!("Missing parameter for `header` expression")
})?,
)?
.to_string(),
Some(pointer) => crate::get_string(
data.pointer(pointer).ok_or_else(|| {
anyhow!(
"Could not find field refered to in parameter `{}`",
pointer
)
})?,
)?,
None => bail!("Invalid expression `{}`", token),
};
command.push_str(&replaced);
trace!("Replace `{}` with: {}", token, replaced);
break;
} else {
command.push('}');
command.push(i);
}
} else {
token.push(i);
}
}
} else {
command.push('{');
command.push(i);
}
} else {
command.push(i);
}
}
Ok(command)
}
}
#[derive(Debug)]
pub struct Hooks {
pub inner: BTreeMap<String, String>,
}
impl Hooks {
pub async fn get_commands(
request: &Request<'_>,
data: Data<'_>,
) -> Result<Self, WebhookeyError> {
let mut buffer = Vec::new();
let size = data
.open(1_i32.megabytes())
.read_to_end(&mut buffer)
.map_err(WebhookeyError::Io)
.await?;
info!("Data of size {} received", size);
let config = request.guard::<&State<Config>>().await.unwrap(); // should never fail
let mut valid = false;
let mut result = BTreeMap::new();
let client_ip = &request
.client_ip()
.unwrap_or(IpAddr::V4(Ipv4Addr::UNSPECIFIED));
let hooks = config.hooks.iter().filter(|(name, hook)| {
if let Some(ip) = &hook.ip_filter {
accept_ip(name, client_ip, ip)
} else {
info!(
"Allow hook `{}` from {}, no IP filter was configured",
&name, &client_ip
);
true
}
});
for (hook_name, hook) in hooks {
let signature = request
.headers()
.get_one(&hook.signature)
.ok_or(WebhookeyError::InvalidSignature)?;
let secrets = hook
.secrets
.iter()
.map(|secret| validate_request(secret, signature, &buffer));
for secret in secrets {
match secret {
Ok(()) => {
trace!("Valid signature found for hook `{}`", hook_name);
valid = true;
let mut data: serde_json::Value =
serde_json::from_slice(&buffer).map_err(WebhookeyError::Serde)?;
match hook.filter.evaluate(request, &data) {
Ok(true) => match hook.get_command(hook_name, request, &mut data) {
Ok(command) => {
info!("Filter for `{}` matched", &hook_name);
result.insert(hook_name.to_string(), command);
break;
}
Err(e) => error!("{}", e),
},
Ok(false) => info!("Filter for `{}` did not match", &hook_name),
Err(error) => {
error!("Could not match filter for `{}`: {}", &hook_name, error)
}
}
}
Err(e) => trace!("Hook `{}` could not validate request: {}", &hook_name, e),
}
}
}
if !valid {
return Err(WebhookeyError::Unauthorized(*client_ip));
}
Ok(Hooks { inner: result })
}
}
#[rocket::async_trait]
impl<'r> FromData<'r> for Hooks {
type Error = WebhookeyError;
async fn from_data(
request: &'r Request<'_>,
data: Data<'r>,
) -> Outcome<Self, (Status, Self::Error), Data<'r>> {
{
request
.guard::<&State<Metrics>>()
.await
.unwrap() // TODO: Check if unwrap need to be fixed
.requests_received
.fetch_add(1, Ordering::Relaxed);
}
match Hooks::get_commands(request, data).await {
Ok(hooks) => {
if hooks.inner.is_empty() {
let client_ip = &request
.client_ip()
.unwrap_or(IpAddr::V4(Ipv4Addr::UNSPECIFIED));
request
.guard::<&State<Metrics>>()
.await
.unwrap() // TODO: Check if unwrap need to be fixed
.hooks_unmatched
.fetch_add(1, Ordering::Relaxed);
warn!("Unmatched hook from {}", &client_ip);
return Failure((Status::NotFound, WebhookeyError::UnmatchedHook(*client_ip)));
}
Success(hooks)
}
Err(WebhookeyError::Unauthorized(e)) => {
error!("{}", WebhookeyError::Unauthorized(e));
request
.guard::<&State<Metrics>>()
.await
.unwrap() // TODO: Check if unwrap need to be fixed
.hooks_forbidden
.fetch_add(1, Ordering::Relaxed);
Failure((Status::Unauthorized, WebhookeyError::Unauthorized(e)))
}
Err(e) => {
error!("{}", e);
request
.guard::<&State<Metrics>>()
.await
.unwrap() // TODO: Check if unwrap need to be fixed
.requests_invalid
.fetch_add(1, Ordering::Relaxed);
Failure((Status::BadRequest, e))
}
}
}
}
#[post("/", format = "json", data = "<hooks>")]
pub async fn receive_hook<'a>(
address: SocketAddr,
hooks: Hooks,
metrics: &State<Metrics>,
) -> Status {
info!("Post request received from: {}", address);
hooks.inner.iter().for_each(|(name, command)| {
info!("Execute `{}` from hook `{}`", &command, &name);
match run_script::run(command, &vec![], &ScriptOptions::new()) {
Ok((status, stdout, stderr)) => {
info!("Command `{}` exited with return code: {}", &command, status);
trace!("Output of command `{}` on stdout: {:?}", &command, &stdout);
debug!("Output of command `{}` on stderr: {:?}", &command, &stderr);
metrics.commands_executed.fetch_add(1, Ordering::Relaxed);
let _ = match status {
0 => metrics.commands_successful.fetch_add(1, Ordering::Relaxed),
_ => metrics.commands_failed.fetch_add(1, Ordering::Relaxed),
};
}
Err(e) => {
error!("Execution of `{}` failed: {}", &command, e);
metrics
.commands_execution_failed
.fetch_add(1, Ordering::Relaxed);
}
}
metrics.hooks_successful.fetch_add(1, Ordering::Relaxed);
});
Status::Ok
}
#[cfg(test)]
mod tests {
use super::*;
use crate::{
config::MetricsConfig,
filters::{AddrType, FilterType, HeaderFilter, JsonFilter},
hooks::Hook,
Metrics,
};
use regex::Regex;
use rocket::{
http::{ContentType, Header, Status},
local::asynchronous::Client,
routes,
};
use serde_json::json;
use std::collections::BTreeMap;
#[rocket::async_test]
async fn secret() {
let mut hooks = BTreeMap::new();
hooks.insert(
"test_hook".to_string(),
Hook {
command: "".to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: None,
secrets: vec!["valid".to_string()],
filter: FilterType::JsonFilter(JsonFilter {
pointer: "*".to_string(),
regex: Regex::new(".*").unwrap(),
}),
},
);
let config = Config {
metrics: None,
hooks: hooks,
};
let rocket = rocket::build()
.mount("/", routes![receive_hook])
.manage(config)
.manage(Metrics::default());
let client = Client::tracked(rocket).await.unwrap();
let response = client
.post("/")
.header(Header::new(
"X-Gitea-Signature",
"28175a0035f637f3cbb85afee9f9d319631580e7621cf790cd16ca063a2f820e",
))
.header(ContentType::JSON)
.remote("127.0.0.1:8000".parse().unwrap())
.body(&serde_json::to_string(&json!({ "foo": "bar" })).unwrap())
.dispatch();
assert_eq!(response.await.status(), Status::NotFound);
let response = client
.post("/")
.header(Header::new("X-Gitea-Signature", "beef"))
.header(ContentType::JSON)
.remote("127.0.0.1:8000".parse().unwrap())
.body(&serde_json::to_string(&json!({ "foo": "bar" })).unwrap())
.dispatch();
assert_eq!(response.await.status(), Status::Unauthorized);
let response = client
.post("/")
.header(Header::new(
"X-Gitea-Signature",
"c5c315d76318362ec129ca629b50b626bba09ad3d7ba4cc0f4c0afe4a90537a0",
))
.header(ContentType::JSON)
.remote("127.0.0.1:8000".parse().unwrap())
.body(r#"{ "not_secret": "invalid" "#)
.dispatch();
assert_eq!(response.await.status(), Status::BadRequest);
let response = client
.post("/")
.header(Header::new("X-Gitea-Signature", "foobar"))
.header(ContentType::JSON)
.remote("127.0.0.1:8000".parse().unwrap())
.dispatch();
assert_eq!(response.await.status(), Status::Unauthorized);
}
#[rocket::async_test]
async fn parse_command_request() {
let mut hooks = BTreeMap::new();
hooks.insert(
"test_hook0".to_string(),
Hook {
command:
"/usr/bin/echo {{ /repository/full_name }} --foo {{ /pull_request/base/ref }}"
.to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: None,
secrets: vec!["valid".to_string()],
filter: FilterType::JsonFilter(JsonFilter {
pointer: "/foo".to_string(),
regex: Regex::new("bar").unwrap(),
}),
},
);
hooks.insert(
"test_hook2".to_string(),
Hook {
command: "/usr/bin/echo {{ /repository/full_name }} {{ /pull_request/base/ref }}"
.to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: None,
secrets: vec!["valid".to_string()],
filter: FilterType::JsonFilter(JsonFilter {
pointer: "/foo".to_string(),
regex: Regex::new("bar").unwrap(),
}),
},
);
hooks.insert(
"test_hook3".to_string(),
Hook {
command: "/usr/bin/echo {{ /repository/full_name }} {{ /pull_request/base/ref }}"
.to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: None,
secrets: vec!["valid".to_string()],
filter: FilterType::Not(Box::new(FilterType::JsonFilter(JsonFilter {
pointer: "/foobar".to_string(),
regex: Regex::new("bar").unwrap(),
}))),
},
);
let config = Config {
metrics: None,
hooks: hooks,
};
let rocket = rocket::build()
.mount("/", routes![receive_hook])
.manage(config)
.manage(Metrics::default());
let client = Client::tracked(rocket).await.unwrap();
let response = client
.post("/")
.header(Header::new(
"X-Gitea-Signature",
"693b733871ecb684651a813c82936df683c9e4a816581f385353e06170545400",
))
.header(ContentType::JSON)
.remote("127.0.0.1:8000".parse().unwrap())
.body(
&serde_json::to_string(&json!({
"foo": "bar",
"repository": {
"full_name": "keith"
},
"pull_request": {
"base": {
"ref": "main"
}
}
}))
.unwrap(),
)
.dispatch();
assert_eq!(response.await.status(), Status::Ok);
}
#[rocket::async_test]
async fn parse_invalid_command_request() {
let mut hooks = BTreeMap::new();
hooks.insert(
"test_hook".to_string(),
Hook {
command: "/usr/bin/echo {{ /repository/full }} {{ /pull_request/base/ref }}"
.to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: None,
secrets: vec!["valid".to_string()],
filter: FilterType::JsonFilter(JsonFilter {
pointer: "/foo".to_string(),
regex: Regex::new("bar").unwrap(),
}),
},
);
let config = Config {
metrics: None,
hooks: hooks,
};
let rocket = rocket::build()
.mount("/", routes![receive_hook])
.manage(config)
.manage(Metrics::default());
let client = Client::tracked(rocket).await.unwrap();
let response = client
.post("/")
.header(Header::new(
"X-Gitea-Signature",
"693b733871ecb684651a813c82936df683c9e4a816581f385353e06170545400",
))
.header(ContentType::JSON)
.remote("127.0.0.1:8000".parse().unwrap())
.body(
&serde_json::to_string(&json!({
"foo": "bar",
"repository": {
"full_name": "keith"
},
"pull_request": {
"base": {
"ref": "main"
}
}
}))
.unwrap(),
)
.dispatch();
assert_eq!(response.await.status(), Status::NotFound);
}
#[test]
fn parse_command() {
let mut headers = HeaderMap::new();
headers.add_raw("X-Gitea-Event", "something");
assert_eq!(
Hook::replace_parameters("command", &headers, &serde_json::Value::Null).unwrap(),
"command"
);
assert_eq!(
Hook::replace_parameters(" command", &headers, &serde_json::Value::Null).unwrap(),
" command"
);
assert_eq!(
Hook::replace_parameters("command ", &headers, &serde_json::Value::Null).unwrap(),
"command "
);
assert_eq!(
Hook::replace_parameters(" command ", &headers, &serde_json::Value::Null)
.unwrap(),
" command "
);
assert_eq!(
Hook::replace_parameters("command command ", &headers, &serde_json::Value::Null)
.unwrap(),
"command command "
);
assert_eq!(
Hook::replace_parameters("{{ /foo }} command", &headers, &json!({ "foo": "bar" }))
.unwrap(),
"bar command"
);
assert_eq!(
Hook::replace_parameters(
" command {{ /foo }} ",
&headers,
&json!({ "foo": "bar" })
)
.unwrap(),
" command bar "
);
assert_eq!(
Hook::replace_parameters(
"{{ /foo }} command{{/field1/foo}}",
&headers,
&json!({ "foo": "bar", "field1": { "foo": "baz" } })
)
.unwrap(),
"bar commandbaz"
);
assert_eq!(
Hook::replace_parameters(
" command {{ /foo }} ",
&headers,
&json!({ "foo": "bar" })
)
.unwrap(),
" command bar "
);
assert_eq!(
Hook::replace_parameters(
" {{ /field1/foo }} command",
&headers,
&json!({ "field1": { "foo": "bar" } })
)
.unwrap(),
" bar command"
);
assert_eq!(
Hook::replace_parameters(
" {{ header X-Gitea-Event }} command",
&headers,
&json!({ "field1": { "foo": "bar" } })
)
.unwrap(),
" something command"
);
assert_eq!(
Hook::replace_parameters(
" {{ header X-Gitea-Event }} {{ /field1/foo }} command",
&headers,
&json!({ "field1": { "foo": "bar" } })
)
.unwrap(),
" something bar command"
);
assert_eq!(
Hook::replace_parameters(
" {{ header X-Gitea-Event }} {{ /field1/foo }} {{ /field1/bar }} {{ /field2/foo }} --command{{ /cmd }}",
&headers,
&json!({ "field1": { "foo": "bar", "bar": "baz" }, "field2": { "foo": "qux" }, "cmd": " else"})
)
.unwrap(),
" something bar baz qux --command else"
);
}
#[test]
fn parse_config() {
let config: Config = serde_yaml::from_str(
r#"---
hooks:
hook1:
command: /usr/bin/local/script_xy.sh {{ /field2/foo }} asdfasdf
signature: X-Gitea-Signature
ip_filter:
allow:
- 127.0.0.1/31
secrets:
- secret_key_01
- secret_key_02
filter:
json:
pointer: /ref
regex: refs/heads/master
hook2:
command: /usr/bin/local/script_xy.sh asdfasdf
signature: X-Gitea-Signature
secrets:
- secret_key_01
- secret_key_02
filter:
and:
- json:
pointer: /ref
regex: refs/heads/master
- header:
field: X-Gitea-Signature
regex: f6e5fe4fe37df76629112d55cc210718b6a55e7e"#,
)
.unwrap();
assert_eq!(
serde_yaml::to_string(&config).unwrap(),
serde_yaml::to_string(&Config {
metrics: None,
hooks: BTreeMap::from([
(
"hook1".to_string(),
Hook {
command: "/usr/bin/local/script_xy.sh {{ /field2/foo }} asdfasdf"
.to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: Some(IpFilter::Allow(vec![AddrType::IpNet(
"127.0.0.1/31".parse().unwrap()
)])),
secrets: vec!["secret_key_01".to_string(), "secret_key_02".to_string()],
filter: FilterType::JsonFilter(JsonFilter {
pointer: "/ref".to_string(),
regex: Regex::new("refs/heads/master").unwrap(),
}),
}
),
(
"hook2".to_string(),
Hook {
command: "/usr/bin/local/script_xy.sh asdfasdf".to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: None,
secrets: vec!["secret_key_01".to_string(), "secret_key_02".to_string()],
filter: FilterType::And(vec![
FilterType::JsonFilter(JsonFilter {
pointer: "/ref".to_string(),
regex: Regex::new("refs/heads/master").unwrap(),
}),
FilterType::HeaderFilter(HeaderFilter {
field: "X-Gitea-Signature".to_string(),
regex: Regex::new("f6e5fe4fe37df76629112d55cc210718b6a55e7e")
.unwrap(),
}),
]),
}
)
])
})
.unwrap()
);
let config: Config = serde_yaml::from_str(
r#"---
metrics:
enabled: true
hooks:
hook1:
command: /usr/bin/local/script_xy.sh {{ /field2/foo }} asdfasdf
signature: X-Gitea-Signature
ip_filter:
allow:
- 127.0.0.1/31
secrets:
- secret_key_01
- secret_key_02
filter:
json:
pointer: /ref
regex: refs/heads/master"#,
)
.unwrap();
assert_eq!(
serde_yaml::to_string(&config).unwrap(),
serde_yaml::to_string(&Config {
metrics: Some(MetricsConfig {
enabled: true,
ip_filter: None
}),
hooks: BTreeMap::from([(
"hook1".to_string(),
Hook {
command: "/usr/bin/local/script_xy.sh {{ /field2/foo }} asdfasdf"
.to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: Some(IpFilter::Allow(vec![AddrType::IpNet(
"127.0.0.1/31".parse().unwrap()
)])),
secrets: vec!["secret_key_01".to_string(), "secret_key_02".to_string()],
filter: FilterType::JsonFilter(JsonFilter {
pointer: "/ref".to_string(),
regex: Regex::new("refs/heads/master").unwrap(),
}),
}
),])
})
.unwrap()
);
}
}

941
src/main.rs
View file

@ -1,43 +1,454 @@
mod cli;
mod config;
mod filters;
mod hooks;
mod metrics;
use crate::{cli::Opts, config::Config, metrics::Metrics};
use anyhow::Result;
use anyhow::{anyhow, bail, Result};
use clap::Parser;
use log::{debug, error, trace};
use rocket::routes;
use std::{fs::File, io::BufReader, net::IpAddr};
use thiserror::Error;
use hmac::{Hmac, Mac, NewMac};
use log::{debug, error, info, trace, warn};
use rocket::{
data::{FromData, ToByteUnit},
futures::TryFutureExt,
get,
http::{HeaderMap, Status},
outcome::Outcome::{self, Failure, Success},
post, routes,
tokio::io::AsyncReadExt,
Data, Request, State,
};
use run_script::ScriptOptions;
use serde::{Deserialize, Serialize};
use sha2::Sha256;
#[derive(Debug, Error)]
pub enum WebhookeyError {
#[error("Could not extract signature from header")]
InvalidSignature,
#[error("Unauthorized request from `{0}`")]
Unauthorized(IpAddr),
#[error("Unmatched hook from `{0}`")]
UnmatchedHook(IpAddr),
#[error("Could not evaluate filter request")]
InvalidFilter,
#[error("IO Error")]
Io(#[from] std::io::Error),
#[error("Serde Error")]
Serde(#[from] serde_json::Error),
use std::{
collections::BTreeMap,
fs::File,
io::BufReader,
net::{IpAddr, Ipv4Addr, SocketAddr},
sync::atomic::{AtomicUsize, Ordering},
};
mod cli;
mod webhooks;
use crate::{
cli::Opts,
webhooks::{FilterType, IpFilter, WebhookeyError},
};
#[derive(Debug, Default)]
struct WebhookeyMetrics {
requests_received: AtomicUsize,
requests_invalid: AtomicUsize,
hooks_successful: AtomicUsize,
hooks_forbidden: AtomicUsize,
hooks_unmatched: AtomicUsize,
commands_executed: AtomicUsize,
commands_execution_failed: AtomicUsize,
commands_successful: AtomicUsize,
commands_failed: AtomicUsize,
}
pub fn get_string(data: &serde_json::Value) -> Result<String, WebhookeyError> {
match &data {
serde_json::Value::Bool(bool) => Ok(bool.to_string()),
serde_json::Value::Number(number) => Ok(number.to_string()),
serde_json::Value::String(string) => Ok(string.as_str().to_string()),
x => {
error!("Could not get string from: {:?}", x);
unimplemented!()
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
struct MetricsConfig {
enabled: bool,
ip_filter: Option<IpFilter>,
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
struct Config {
metrics: Option<MetricsConfig>,
hooks: BTreeMap<String, Hook>,
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
struct Hook {
command: String,
signature: String,
ip_filter: Option<IpFilter>,
secrets: Vec<String>,
filter: FilterType,
}
impl Hook {
fn get_command(
&self,
hook_name: &str,
request: &Request,
data: &mut serde_json::Value,
) -> Result<String> {
debug!("Replacing parameters for command of hook `{}`", hook_name);
Hook::replace_parameters(&self.command, request.headers(), data)
}
fn replace_parameters(
input: &str,
headers: &HeaderMap,
data: &serde_json::Value,
) -> Result<String> {
let mut command = String::new();
let command_template = &mut input.chars();
while let Some(i) = command_template.next() {
if i == '{' {
if let Some('{') = command_template.next() {
let mut token = String::new();
while let Some(i) = command_template.next() {
if i == '}' {
if let Some('}') = command_template.next() {
let expr = token.trim().split(' ').collect::<Vec<&str>>();
let replaced = match expr.get(0) {
Some(&"header") => get_header_field(
headers,
expr.get(1).ok_or_else(|| {
anyhow!("Missing parameter for `header` expression")
})?,
)?
.to_string(),
Some(pointer) => webhooks::get_string(
data.pointer(pointer).ok_or_else(|| {
anyhow!(
"Could not find field refered to in parameter `{}`",
pointer
)
})?,
)?,
None => bail!("Missing expression in variable `{}`", token),
};
command.push_str(&replaced);
trace!("Replace `{}` with: {}", token, replaced);
break;
} else {
command.push('}');
command.push(i);
}
} else {
token.push(i);
}
}
} else {
command.push('{');
command.push(i);
}
} else {
command.push(i);
}
}
Ok(command)
}
}
#[derive(Debug)]
struct Hooks {
inner: BTreeMap<String, String>,
}
impl Hooks {
async fn get_commands(request: &Request<'_>, data: Data<'_>) -> Result<Self, WebhookeyError> {
let mut buffer = Vec::new();
let size = data
.open(256_i32.kilobytes())
.read_to_end(&mut buffer)
.map_err(WebhookeyError::Io)
.await?;
info!("Data of size {} received", size);
let config = request.guard::<&State<Config>>().await.unwrap(); // should never fail
let mut valid = false;
let mut result = BTreeMap::new();
let client_ip = &request
.client_ip()
.unwrap_or(IpAddr::V4(Ipv4Addr::UNSPECIFIED));
let hooks = config.hooks.iter().filter(|(name, hook)| {
if let Some(ip) = &hook.ip_filter {
accept_ip(name, client_ip, ip)
} else {
info!(
"Allow hook `{}` from {}, no IP filter was configured",
&name, &client_ip
);
true
}
});
for (hook_name, hook) in hooks {
let signature = request
.headers()
.get_one(&hook.signature)
.ok_or(WebhookeyError::InvalidSignature)?;
let secrets = hook
.secrets
.iter()
.map(|secret| validate_request(secret, signature, &buffer));
for secret in secrets {
match secret {
Ok(()) => {
trace!("Valid signature found for hook `{}`", hook_name);
valid = true;
let mut data: serde_json::Value =
serde_json::from_slice(&buffer).map_err(WebhookeyError::Serde)?;
match hook.filter.evaluate(&data) {
Ok(true) => match hook.get_command(hook_name, request, &mut data) {
Ok(command) => {
info!("Filter for `{}` matched", &hook_name);
result.insert(hook_name.to_string(), command);
break;
}
Err(e) => error!("{}", e),
},
Ok(false) => info!("Filter for `{}` did not match", &hook_name),
Err(error) => {
error!("Could not match filter for `{}`: {}", &hook_name, error)
}
}
}
Err(e) => trace!("Hook `{}` could not validate request: {}", &hook_name, e),
}
}
}
if !valid {
return Err(WebhookeyError::Unauthorized(*client_ip));
}
Ok(Hooks { inner: result })
}
}
fn accept_ip(hook_name: &str, client_ip: &IpAddr, ip: &IpFilter) -> bool {
if ip.validate(client_ip) {
info!("Allow hook `{}` from {}", &hook_name, &client_ip);
return true;
}
warn!("Deny hook `{}` from {}", &hook_name, &client_ip);
false
}
fn validate_request(secret: &str, signature: &str, data: &[u8]) -> Result<()> {
let mut mac = Hmac::<Sha256>::new_from_slice(secret.as_bytes())
.map_err(|e| anyhow!("Could not create hasher with secret: {}", e))?;
mac.update(data);
let raw_signature = hex::decode(signature.as_bytes())?;
mac.verify(&raw_signature).map_err(|e| anyhow!("{}", e))
}
fn get_header_field<'a>(headers: &'a HeaderMap, param: &str) -> Result<&'a str> {
headers
.get_one(param)
.ok_or_else(|| anyhow!("Could not extract event parameter from header"))
}
fn get_config() -> Result<File> {
// Look for config in CWD..
if let Ok(config) = File::open("config.yml") {
info!("Loading configuration from `./config.yml`");
return Ok(config);
}
// ..look for user path config..
if let Some(mut path) = dirs::config_dir() {
path.push("webhookey/config.yml");
if let Ok(config) = File::open(&path) {
info!(
"Loading configuration from `{}`",
path.to_str().unwrap_or("<path unprintable>"),
);
return Ok(config);
}
}
// ..look for systemwide config..
if let Ok(config) = File::open("/etc/webhookey/config.yml") {
info!("Loading configuration from `/etc/webhookey/config.yml`");
return Ok(config);
}
// ..you had your chance.
bail!("No configuration file found.");
}
fn get_metrics(metrics: &WebhookeyMetrics) -> String {
format!(
r"# HELP webhookey_requests_received Number of requests received
# TYPE webhookey_requests_received gauge
webhookey_requests_received {}
# HELP webhookey_requests_invalid Number of invalid requests received
# TYPE webhookey_requests_invalid gauge
webhookey_requests_invalid {}
# HELP webhookey_hooks_successful Number of successfully executed hooks
# TYPE webhookey_hooks_successful gauge
webhookey_hooks_sucessful {}
# HELP webhookey_hooks_forbidden Number of forbidden requests
# TYPE webhookey_hooks_forbidden gauge
webhookey_hooks_forbidden {}
# HELP webhookey_hooks_unmatched Number of unmatched requests
# TYPE webhookey_hooks_unmatched gauge
webhookey_hooks_unmatched {}
# HELP webhookey_commands_executed Number of commands executed
# TYPE webhookey_commands_executed gauge
webhookey_commands_executed {}
# HELP webhookey_commands_execution_failed Number of commands failed to execute
# TYPE webhookey_commands_execution_failed gauge
webhookey_commands_execution_failed {}
# HELP webhookey_commands_successful Number of executed commands returning return code 0
# TYPE webhookey_commands_successful gauge
webhookey_commands_successful {}
# HELP webhookey_commands_failed Number of executed commands returning different return code than 0
# TYPE webhookey_commands_failed gauge
webhookey_commands_failed {}
",
metrics.requests_received.load(Ordering::Relaxed),
metrics.requests_invalid.load(Ordering::Relaxed),
metrics.hooks_successful.load(Ordering::Relaxed),
metrics.hooks_forbidden.load(Ordering::Relaxed),
metrics.hooks_unmatched.load(Ordering::Relaxed),
metrics.commands_executed.load(Ordering::Relaxed),
metrics.commands_execution_failed.load(Ordering::Relaxed),
metrics.commands_successful.load(Ordering::Relaxed),
metrics.commands_failed.load(Ordering::Relaxed),
)
}
#[rocket::async_trait]
impl<'r> FromData<'r> for Hooks {
type Error = WebhookeyError;
async fn from_data(
request: &'r Request<'_>,
data: Data<'r>,
) -> Outcome<Self, (Status, Self::Error), Data<'r>> {
{
request
.guard::<&State<WebhookeyMetrics>>()
.await
.unwrap() // TODO: Check if unwrap need to be fixed
.requests_received
.fetch_add(1, Ordering::Relaxed);
}
match Hooks::get_commands(request, data).await {
Ok(hooks) => {
if hooks.inner.is_empty() {
let client_ip = &request
.client_ip()
.unwrap_or(IpAddr::V4(Ipv4Addr::UNSPECIFIED));
request
.guard::<&State<WebhookeyMetrics>>()
.await
.unwrap() // TODO: Check if unwrap need to be fixed
.hooks_unmatched
.fetch_add(1, Ordering::Relaxed);
warn!("Unmatched hook from {}", &client_ip);
return Failure((Status::NotFound, WebhookeyError::UnmatchedHook(*client_ip)));
}
Success(hooks)
}
Err(WebhookeyError::Unauthorized(e)) => {
error!("{}", WebhookeyError::Unauthorized(e));
request
.guard::<&State<WebhookeyMetrics>>()
.await
.unwrap() // TODO: Check if unwrap need to be fixed
.hooks_forbidden
.fetch_add(1, Ordering::Relaxed);
Failure((Status::Unauthorized, WebhookeyError::Unauthorized(e)))
}
Err(e) => {
error!("{}", e);
request
.guard::<&State<WebhookeyMetrics>>()
.await
.unwrap() // TODO: Check if unwrap need to be fixed
.requests_invalid
.fetch_add(1, Ordering::Relaxed);
Failure((Status::BadRequest, e))
}
}
}
}
#[post("/", format = "json", data = "<hooks>")]
async fn receive_hook<'a>(
address: SocketAddr,
hooks: Hooks,
metrics: &State<WebhookeyMetrics>,
) -> Status {
info!("Post request received from: {}", address);
hooks.inner.iter().for_each(|(name, command)| {
info!("Execute `{}` from hook `{}`", &command, &name);
match run_script::run(command, &vec![], &ScriptOptions::new()) {
Ok((status, stdout, stderr)) => {
info!("Command `{}` exited with return code: {}", &command, status);
trace!("Output of command `{}` on stdout: {:?}", &command, &stdout);
debug!("Output of command `{}` on stderr: {:?}", &command, &stderr);
metrics.commands_executed.fetch_add(1, Ordering::Relaxed);
let _ = match status {
0 => metrics.commands_successful.fetch_add(1, Ordering::Relaxed),
_ => metrics.commands_failed.fetch_add(1, Ordering::Relaxed),
};
}
Err(e) => {
error!("Execution of `{}` failed: {}", &command, e);
metrics
.commands_execution_failed
.fetch_add(1, Ordering::Relaxed);
}
}
});
Status::Ok
}
#[get("/metrics")]
async fn metrics(
address: SocketAddr,
metrics: &State<WebhookeyMetrics>,
config: &State<Config>,
) -> Option<String> {
if let Some(metrics_config) = &config.metrics {
if metrics_config.enabled {
if let Some(filter) = &metrics_config.ip_filter {
if filter.validate(&address.ip()) {
return Some(get_metrics(metrics));
}
} else {
return Some(get_metrics(metrics));
}
}
}
warn!("Forbidden request for metrics: {:?}", address);
None
}
#[rocket::main]
@ -48,7 +459,7 @@ async fn main() -> Result<()> {
let config: Config = match cli.config {
Some(config) => serde_yaml::from_reader(BufReader::new(File::open(config)?))?,
_ => serde_yaml::from_reader(BufReader::new(config::get_config()?))?,
_ => serde_yaml::from_reader(BufReader::new(get_config()?))?,
};
trace!("Parsed configuration:\n{}", serde_yaml::to_string(&config)?);
@ -60,11 +471,467 @@ async fn main() -> Result<()> {
}
rocket::build()
.mount("/", routes![hooks::receive_hook, metrics::metrics])
.mount("/", routes![receive_hook, metrics])
.manage(config)
.manage(Metrics::default())
.manage(WebhookeyMetrics::default())
.launch()
.await?;
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use crate::webhooks::{AddrType, JsonFilter};
use rocket::{
http::{ContentType, Header},
local::asynchronous::Client,
};
use serde_json::json;
#[rocket::async_test]
async fn secret() {
let mut hooks = BTreeMap::new();
hooks.insert(
"test_hook".to_string(),
Hook {
command: "".to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: None,
secrets: vec!["valid".to_string()],
filter: FilterType::JsonFilter(JsonFilter {
pointer: "*".to_string(),
regex: "*".to_string(),
}),
},
);
let config = Config {
metrics: None,
hooks: hooks,
};
let rocket = rocket::build()
.mount("/", routes![receive_hook])
.manage(config)
.manage(WebhookeyMetrics::default());
let client = Client::tracked(rocket).await.unwrap();
let response = client
.post("/")
.header(Header::new(
"X-Gitea-Signature",
"28175a0035f637f3cbb85afee9f9d319631580e7621cf790cd16ca063a2f820e",
))
.header(ContentType::JSON)
.remote("127.0.0.1:8000".parse().unwrap())
.body(&serde_json::to_string(&json!({ "foo": "bar" })).unwrap())
.dispatch();
assert_eq!(response.await.status(), Status::NotFound);
let response = client
.post("/")
.header(Header::new("X-Gitea-Signature", "beef"))
.header(ContentType::JSON)
.remote("127.0.0.1:8000".parse().unwrap())
.body(&serde_json::to_string(&json!({ "foo": "bar" })).unwrap())
.dispatch();
assert_eq!(response.await.status(), Status::Unauthorized);
let response = client
.post("/")
.header(Header::new(
"X-Gitea-Signature",
"c5c315d76318362ec129ca629b50b626bba09ad3d7ba4cc0f4c0afe4a90537a0",
))
.header(ContentType::JSON)
.remote("127.0.0.1:8000".parse().unwrap())
.body(r#"{ "not_secret": "invalid" "#)
.dispatch();
assert_eq!(response.await.status(), Status::BadRequest);
let response = client
.post("/")
.header(Header::new("X-Gitea-Signature", "foobar"))
.header(ContentType::JSON)
.remote("127.0.0.1:8000".parse().unwrap())
.dispatch();
assert_eq!(response.await.status(), Status::Unauthorized);
}
#[test]
fn parse_command() {
let mut headers = HeaderMap::new();
headers.add_raw("X-Gitea-Event", "something");
assert_eq!(
Hook::replace_parameters("command", &headers, &serde_json::Value::Null).unwrap(),
"command"
);
assert_eq!(
Hook::replace_parameters(" command", &headers, &serde_json::Value::Null).unwrap(),
" command"
);
assert_eq!(
Hook::replace_parameters("command ", &headers, &serde_json::Value::Null).unwrap(),
"command "
);
assert_eq!(
Hook::replace_parameters(" command ", &headers, &serde_json::Value::Null)
.unwrap(),
" command "
);
assert_eq!(
Hook::replace_parameters("command command ", &headers, &serde_json::Value::Null)
.unwrap(),
"command command "
);
assert_eq!(
Hook::replace_parameters("{{ /foo }} command", &headers, &json!({ "foo": "bar" }))
.unwrap(),
"bar command"
);
assert_eq!(
Hook::replace_parameters(
" command {{ /foo }} ",
&headers,
&json!({ "foo": "bar" })
)
.unwrap(),
" command bar "
);
assert_eq!(
Hook::replace_parameters(
"{{ /foo }} command{{/field1/foo}}",
&headers,
&json!({ "foo": "bar", "field1": { "foo": "baz" } })
)
.unwrap(),
"bar commandbaz"
);
assert_eq!(
Hook::replace_parameters(
" command {{ /foo }} ",
&headers,
&json!({ "foo": "bar" })
)
.unwrap(),
" command bar "
);
assert_eq!(
Hook::replace_parameters(
" {{ /field1/foo }} command",
&headers,
&json!({ "field1": { "foo": "bar" } })
)
.unwrap(),
" bar command"
);
assert_eq!(
Hook::replace_parameters(
" {{ header X-Gitea-Event }} command",
&headers,
&json!({ "field1": { "foo": "bar" } })
)
.unwrap(),
" something command"
);
assert_eq!(
Hook::replace_parameters(
" {{ header X-Gitea-Event }} {{ /field1/foo }} command",
&headers,
&json!({ "field1": { "foo": "bar" } })
)
.unwrap(),
" something bar command"
);
assert_eq!(
Hook::replace_parameters(
" {{ header X-Gitea-Event }} {{ /field1/foo }} {{ /field1/bar }} {{ /field2/foo }} --command{{ /cmd }}",
&headers,
&json!({ "field1": { "foo": "bar", "bar": "baz" }, "field2": { "foo": "qux" }, "cmd": " else"})
)
.unwrap(),
" something bar baz qux --command else"
);
}
#[rocket::async_test]
async fn parse_command_request() {
let mut hooks = BTreeMap::new();
hooks.insert(
"test_hook".to_string(),
Hook {
command:
"/usr/bin/echo {{ /repository/full_name }} --foo {{ /pull_request/base/ref }}"
.to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: None,
secrets: vec!["valid".to_string()],
filter: FilterType::JsonFilter(JsonFilter {
pointer: "/foo".to_string(),
regex: "bar".to_string(),
}),
},
);
hooks.insert(
"test_hook".to_string(),
Hook {
command: "/usr/bin/echo {{ /repository/full_name }} {{ /pull_request/base/ref }}"
.to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: None,
secrets: vec!["valid".to_string()],
filter: FilterType::JsonFilter(JsonFilter {
pointer: "/foo".to_string(),
regex: "bar".to_string(),
}),
},
);
let config = Config {
metrics: None,
hooks: hooks,
};
let rocket = rocket::build()
.mount("/", routes![receive_hook])
.manage(config)
.manage(WebhookeyMetrics::default());
let client = Client::tracked(rocket).await.unwrap();
let response = client
.post("/")
.header(Header::new(
"X-Gitea-Signature",
"693b733871ecb684651a813c82936df683c9e4a816581f385353e06170545400",
))
.header(ContentType::JSON)
.remote("127.0.0.1:8000".parse().unwrap())
.body(
&serde_json::to_string(&json!({
"foo": "bar",
"repository": {
"full_name": "keith"
},
"pull_request": {
"base": {
"ref": "main"
}
}
}))
.unwrap(),
)
.dispatch();
assert_eq!(response.await.status(), Status::Ok);
}
#[rocket::async_test]
async fn parse_invalid_command_request() {
let mut hooks = BTreeMap::new();
hooks.insert(
"test_hook".to_string(),
Hook {
command: "/usr/bin/echo {{ /repository/full }} {{ /pull_request/base/ref }}"
.to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: None,
secrets: vec!["valid".to_string()],
filter: FilterType::JsonFilter(JsonFilter {
pointer: "/foo".to_string(),
regex: "bar".to_string(),
}),
},
);
let config = Config {
metrics: None,
hooks: hooks,
};
let rocket = rocket::build()
.mount("/", routes![receive_hook])
.manage(config)
.manage(WebhookeyMetrics::default());
let client = Client::tracked(rocket).await.unwrap();
let response = client
.post("/")
.header(Header::new(
"X-Gitea-Signature",
"693b733871ecb684651a813c82936df683c9e4a816581f385353e06170545400",
))
.header(ContentType::JSON)
.remote("127.0.0.1:8000".parse().unwrap())
.body(
&serde_json::to_string(&json!({
"foo": "bar",
"repository": {
"full_name": "keith"
},
"pull_request": {
"base": {
"ref": "main"
}
}
}))
.unwrap(),
)
.dispatch();
assert_eq!(response.await.status(), Status::NotFound);
}
#[test]
fn parse_config() {
let config: Config = serde_yaml::from_str(
r#"---
hooks:
hook1:
command: /usr/bin/local/script_xy.sh {{ /field2/foo }} asdfasdf
signature: X-Gitea-Signature
ip_filter:
allow:
- 127.0.0.1/31
secrets:
- secret_key_01
- secret_key_02
filter:
json:
pointer: /ref
regex: refs/heads/master
hook2:
command: /usr/bin/local/script_xy.sh asdfasdf
signature: X-Gitea-Signature
secrets:
- secret_key_01
- secret_key_02
filter:
and:
- json:
pointer: /ref
regex: refs/heads/master
- json:
pointer: /after
regex: f6e5fe4fe37df76629112d55cc210718b6a55e7e"#,
)
.unwrap();
assert_eq!(
serde_yaml::to_string(&config).unwrap(),
serde_yaml::to_string(&Config {
metrics: None,
hooks: BTreeMap::from([
(
"hook1".to_string(),
Hook {
command: "/usr/bin/local/script_xy.sh {{ /field2/foo }} asdfasdf"
.to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: Some(IpFilter::Allow(vec![AddrType::IpNet(
"127.0.0.1/31".parse().unwrap()
)])),
secrets: vec!["secret_key_01".to_string(), "secret_key_02".to_string()],
filter: FilterType::JsonFilter(JsonFilter {
pointer: "/ref".to_string(),
regex: "refs/heads/master".to_string(),
}),
}
),
(
"hook2".to_string(),
Hook {
command: "/usr/bin/local/script_xy.sh asdfasdf".to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: None,
secrets: vec!["secret_key_01".to_string(), "secret_key_02".to_string()],
filter: FilterType::And(vec![
FilterType::JsonFilter(JsonFilter {
pointer: "/ref".to_string(),
regex: "refs/heads/master".to_string(),
}),
FilterType::JsonFilter(JsonFilter {
pointer: "/after".to_string(),
regex: "f6e5fe4fe37df76629112d55cc210718b6a55e7e".to_string(),
}),
]),
}
)
])
})
.unwrap()
);
let config: Config = serde_yaml::from_str(
r#"---
metrics:
enabled: true
hooks:
hook1:
command: /usr/bin/local/script_xy.sh {{ /field2/foo }} asdfasdf
signature: X-Gitea-Signature
ip_filter:
allow:
- 127.0.0.1/31
secrets:
- secret_key_01
- secret_key_02
filter:
json:
pointer: /ref
regex: refs/heads/master"#,
)
.unwrap();
assert_eq!(
serde_yaml::to_string(&config).unwrap(),
serde_yaml::to_string(&Config {
metrics: Some(MetricsConfig {
enabled: true,
ip_filter: None
}),
hooks: BTreeMap::from([(
"hook1".to_string(),
Hook {
command: "/usr/bin/local/script_xy.sh {{ /field2/foo }} asdfasdf"
.to_string(),
signature: "X-Gitea-Signature".to_string(),
ip_filter: Some(IpFilter::Allow(vec![AddrType::IpNet(
"127.0.0.1/31".parse().unwrap()
)])),
secrets: vec!["secret_key_01".to_string(), "secret_key_02".to_string()],
filter: FilterType::JsonFilter(JsonFilter {
pointer: "/ref".to_string(),
regex: "refs/heads/master".to_string(),
}),
}
),])
})
.unwrap()
);
}
}

91
src/metrics.rs
View file

@ -1,91 +0,0 @@
use crate::Config;
use log::warn;
use rocket::{get, State};
use std::{
net::SocketAddr,
sync::atomic::{AtomicUsize, Ordering},
};
#[derive(Debug, Default)]
pub struct Metrics {
pub requests_received: AtomicUsize,
pub requests_invalid: AtomicUsize,
pub hooks_successful: AtomicUsize,
pub hooks_forbidden: AtomicUsize,
pub hooks_unmatched: AtomicUsize,
pub commands_executed: AtomicUsize,
pub commands_execution_failed: AtomicUsize,
pub commands_successful: AtomicUsize,
pub commands_failed: AtomicUsize,
}
#[get("/metrics")]
pub async fn metrics(
address: SocketAddr,
metrics: &State<Metrics>,
config: &State<Config>,
) -> Option<String> {
// Are metrics configured?
if let Some(metrics_config) = &config.metrics {
// Are metrics enabled?
if metrics_config.enabled {
// Is a filter configured?
if let Some(filter) = &metrics_config.ip_filter {
// Does the request match the filter?
if filter.validate(&address.ip()) {
return Some(metrics.get_metrics());
}
} else {
return Some(metrics.get_metrics());
}
}
}
warn!("Forbidden request for metrics: {:?}", address);
None
}
impl Metrics {
fn get_metrics(&self) -> String {
format!(
r"# HELP webhookey_requests_received Number of requests received
# TYPE webhookey_requests_received gauge
webhookey_requests_received {}
# HELP webhookey_requests_invalid Number of invalid requests received
# TYPE webhookey_requests_invalid gauge
webhookey_requests_invalid {}
# HELP webhookey_hooks_successful Number of successfully executed hooks
# TYPE webhookey_hooks_successful gauge
webhookey_hooks_sucessful {}
# HELP webhookey_hooks_forbidden Number of forbidden requests
# TYPE webhookey_hooks_forbidden gauge
webhookey_hooks_forbidden {}
# HELP webhookey_hooks_unmatched Number of unmatched requests
# TYPE webhookey_hooks_unmatched gauge
webhookey_hooks_unmatched {}
# HELP webhookey_commands_executed Number of commands executed
# TYPE webhookey_commands_executed gauge
webhookey_commands_executed {}
# HELP webhookey_commands_execution_failed Number of commands failed to execute
# TYPE webhookey_commands_execution_failed gauge
webhookey_commands_execution_failed {}
# HELP webhookey_commands_successful Number of executed commands returning return code 0
# TYPE webhookey_commands_successful gauge
webhookey_commands_successful {}
# HELP webhookey_commands_failed Number of executed commands returning different return code than 0
# TYPE webhookey_commands_failed gauge
webhookey_commands_failed {}
",
self.requests_received.load(Ordering::Relaxed),
self.requests_invalid.load(Ordering::Relaxed),
self.hooks_successful.load(Ordering::Relaxed),
self.hooks_forbidden.load(Ordering::Relaxed),
self.hooks_unmatched.load(Ordering::Relaxed),
self.commands_executed.load(Ordering::Relaxed),
self.commands_execution_failed.load(Ordering::Relaxed),
self.commands_successful.load(Ordering::Relaxed),
self.commands_failed.load(Ordering::Relaxed),
)
}
}

163
src/webhooks.rs Normal file
View file

@ -0,0 +1,163 @@
use anyhow::Result;
use ipnet::IpNet;
use log::{debug, error, trace};
use regex::Regex;
use serde::{Deserialize, Serialize};
use std::net::IpAddr;
use thiserror::Error;
#[derive(Debug, Error)]
pub enum WebhookeyError {
#[error("Could not extract signature from header")]
InvalidSignature,
#[error("Unauthorized request from `{0}`")]
Unauthorized(IpAddr),
#[error("Unmatched hook from `{0}`")]
UnmatchedHook(IpAddr),
#[error("Could not evaluate filter request")]
InvalidFilter,
#[error("IO Error")]
Io(std::io::Error),
#[error("Serde Error")]
Serde(serde_json::Error),
#[error("Regex Error")]
Regex(regex::Error),
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields, untagged)]
pub enum AddrType {
IpAddr(IpAddr),
IpNet(IpNet),
}
impl AddrType {
pub fn matches(&self, client_ip: &IpAddr) -> bool {
match self {
AddrType::IpAddr(addr) => addr == client_ip,
AddrType::IpNet(net) => net.contains(client_ip),
}
}
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields, rename_all = "lowercase")]
pub enum IpFilter {
Allow(Vec<AddrType>),
Deny(Vec<AddrType>),
}
impl IpFilter {
pub fn validate(&self, client_ip: &IpAddr) -> bool {
match self {
IpFilter::Allow(list) => list.iter().any(|i| i.matches(client_ip)),
IpFilter::Deny(list) => !list.iter().any(|i| i.matches(client_ip)),
}
}
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub struct JsonFilter {
pub pointer: String,
pub regex: String,
}
impl JsonFilter {
pub fn evaluate(&self, data: &serde_json::Value) -> Result<bool, WebhookeyError> {
trace!(
"Matching `{}` on `{}` from received json",
&self.regex,
&self.pointer,
);
let regex = Regex::new(&self.regex).map_err(WebhookeyError::Regex)?;
if let Some(value) = data.pointer(&self.pointer) {
if regex.is_match(&get_string(value)?) {
debug!("Regex `{}` for `{}` matches", &self.regex, &self.pointer);
return Ok(true);
}
}
debug!(
"Regex `{}` for `{}` does not match",
&self.regex, &self.pointer
);
Ok(false)
}
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields, rename_all = "lowercase")]
pub enum FilterType {
And(Vec<FilterType>),
Or(Vec<FilterType>),
#[serde(rename = "json")]
JsonFilter(JsonFilter),
}
impl FilterType {
pub fn evaluate(&self, data: &serde_json::Value) -> Result<bool, WebhookeyError> {
match self {
FilterType::And(filters) => {
let (mut results, mut errors) = (Vec::new(), Vec::new());
filters
.iter()
.map(|filter| filter.evaluate(data))
.for_each(|item| match item {
Ok(o) => results.push(o),
Err(e) => errors.push(e),
});
if errors.is_empty() {
Ok(results.iter().all(|r| *r))
} else {
errors
.iter()
.for_each(|e| error!("Could not evaluate Filter: {}", e));
Err(WebhookeyError::InvalidFilter)
}
}
FilterType::Or(filters) => {
let (mut results, mut errors) = (Vec::new(), Vec::new());
filters
.iter()
.map(|filter| filter.evaluate(data))
.for_each(|item| match item {
Ok(o) => results.push(o),
Err(e) => errors.push(e),
});
if errors.is_empty() {
Ok(results.iter().any(|r| *r))
} else {
errors
.iter()
.for_each(|e| error!("Could not evaluate Filter: {}", e));
Err(WebhookeyError::InvalidFilter)
}
}
// FilterType::HeaderFilter(filter) => todo!(),
FilterType::JsonFilter(filter) => filter.evaluate(data),
}
}
}
pub fn get_string(data: &serde_json::Value) -> Result<String, WebhookeyError> {
match &data {
serde_json::Value::Bool(bool) => Ok(bool.to_string()),
serde_json::Value::Number(number) => Ok(number.to_string()),
serde_json::Value::String(string) => Ok(string.as_str().to_string()),
x => {
error!("Could not get string from: {:?}", x);
unimplemented!()
}
}
}

57
webhookey.1
View file

@ -1,57 +0,0 @@
.TH WEBHOOKEY 1 "26 Nov 2021" "webhookey" "Linux"
.SH NAME
webhookey \- Receive webhooks and act upon them
.SH SYNOPSIS
.B webhookey [OPTIONS] [SUBCOMMAND]
.SH DESCRIPTION
\fBwebhookey\fR receives http(s) requests in form of webhooks. Those
webhooks are matched against configured filters. If a filter matches,
a command (which can also incorporate data contained in the received
header or body) is executed.
.SH OPTIONS
.TP
.BR "-c", " --config " <FILE>
Provide a path to the configuration file.
.TP
.BR "-h", " --help"
Print help information.
.TP
.BR "-V", " --version"
Print version information.
.SH SUBCOMMAND
.TP
.B configtest
- Verifies if the configuration can be parsed without errors.
.TP
.B help
- Print the general help message or the help of the given subcommand(s).
.SH ENVIRONMENT
.TP
.B ROCKET_ADDRESS
The IP address webhookey listens on (default: 127.0.0.1).
.TP
.B ROCKET_PORT
The port webhookey listens on (default: 8000).
.TP
.B ROCKET_WORKERS
The numbers of threads to use (default: CPU core count).
.TP
.B RUST_LOG
Set the Log level, which can be one one of "error", "warn", "info",
"debug", "trace" (default: "error").
.SH EXAMPLES
.PP
webhookey configtest
.RS 4
Return either "Config is OK" and return code 0 or an error description
and return code 1.
.RE
.PP
webhookey
.RS 4
Start webhookey.
.RE
.SH REPORTING BUGS
To report any bugs file an issue at
https://git.onders.org/finga/webhookey/issues/new?template=bug.md or
send an email to <bug-report@onders.org>.