finga/webhookey
1
0
Fork 0
Code Issues 5 Pull requests Projects Releases Wiki Activity

Support so called conjunction filters

Browse source 
This introduces thee so called conjunction filters and therefore
restructures the configuration file. The most obvious changes from an
users perspective are that the `filters` field was renamed to `filter`
and can, from now on, only support a single filter at first
level. Thats why now different filter types are implemented, please
consult the readme for further information on their usage.

To reflect the changes the readme file is updated as well as the
example config file contained in this repository.

This is related to #8
This commit is contained in:
finga 2021-05-31 02:16:22 +02:00
parent 891a8a70ae
commit 43b7fd5625
3 changed files with 185 additions and 96 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

62
README.md
View file

@ -1,6 +1,6 @@
# Webhookey
Webhookey is a webserver listening for requests as for example sent by
gitea's webhooks. Further, Webhookey allows you to specifiy rules
Webhookey is a web server listening for requests as for example sent by
gitea's webhooks. Further, Webhookey allows you to specify rules
which are matched against the data received to trigger certain
actions.
@ -15,7 +15,7 @@ Further, for Rocket we need to have the nightly toolchain installed:
```
### Build Webhookey
The webhookey project can be built for development:
The Webhookey project can be built for development:
``` sh
cargo b
```
@ -27,7 +27,7 @@ or for releasing:
### Install Webhookey
When a Rust toolchain installed you can also install Webhookey
directly without cloning it manualy:
directly without cloning it manually:
``` sh
cargo install --git https://git.onders.org/finga/webhookey.git webhookey
```
@ -51,7 +51,7 @@ you built.
Configuration syntax is YAML and it's paths as well as it's
configuration format is described in the following sections.
### Configuration paths
### Configuration Paths
Following locations are checked for a configuration file:
- `/etc/webhookey/config.yml`
- `<config_dir>/webhookey/config.yml`
@ -72,7 +72,7 @@ consists of the following fields:
source addresses or ranges.
- signature: Name of the HTTP header field containing the signature.
- secrets: List of secrets.
- filters: List of filters.
- filter: Tree of filters.
Example:
```yaml
@ -87,10 +87,18 @@ hooks:
secrets:
- secret_key_01
- secret_key_02
filters:
match_ref:
pointer: /ref
regex: refs/heads/master
filter:
or:
- json:
pointer: /ref
regex: refs/heads/master
- and:
- json:
pointer: /ref
regex: refs/heads/a_branch
- json:
pointer: /after
regex: f6e5fe4fe37df76629112d55cc210718b6a55e7e
```
##### Command
@ -137,17 +145,25 @@ Set the name of the HTTP header field containing the HMAC signature.
Configure a list of secrets to validate the hook.
##### Filter
Each filter must have following fields:
- pointer: pointer to the JSON field according to [RFC
6901](https://tools.ietf.org/html/rfc6901)
- regex: regular expression which has to match the field pointed to by
the pointer
Filter can be either a concrete filter or a conjunction
filter. Concrete filters return either true or false on specific
constraints. Conjunction filters contain lists of filters which are
evaluated and combined based on the type. The result is either used
for parent conjunction filters or, if at the root, used to decide if a
hook should be executed.
# TODOs
## Use `clap` to parse command line arguments
## Configure rocket via config.yml
## Security
### https support
basically supported, but related to "Configure rocket via config.yml".
### Authentication features
## Use proptest or quickcheck for tests of parsers
###### Conjunction Filters
Conjunction filters contain lists of other filters.
- `and`: Logical conjunction.
- `or`: Logical disjunction.
###### Concrete Filters
- `json`:
The `json` filter matches a regular expression on a field from the
received JSON data.
- pointer: Pointer to the JSON field according to [RFC
6901](https://tools.ietf.org/html/rfc6901).
- regex: Regular expression which has to match the field pointed to
by the pointer.

31
config.yml
View file

@ -9,8 +9,8 @@ hooks:
secrets:
- secret_key_01
- secret_key_02
filters:
match_ref:
filter:
json:
pointer: /ref
regex: refs/heads/master
hook2:
@ -22,19 +22,24 @@ hooks:
secrets:
- secret_key_01
- secret_key_02
filters:
match_ref:
pointer: /ref
regex: refs/heads/master
filter:
and:
- json:
pointer: /ref
regex: refs/heads/master
- json:
pointer: /after
regex: f6e5fe4fe37df76629112d55cc210718b6a55e7e
hook3:
command: /usr/bin/local/script_xyz.sh
signature: X-Gitea-Signature
secrets:
- secret_key03
filters:
match_ref:
pointer: /ref
regex: refs/heads/master
match_after:
pointer: /after
regex: f6e5fe4fe37df76629112d55cc210718b6a55e7e
filter:
or:
- json:
pointer: /ref
regex: refs/heads/master
- json:
pointer: /after
regex: f6e5fe4fe37df76629112d55cc210718b6a55e7e

188
src/main.rs
View file

@ -71,6 +71,95 @@ struct Config {
hooks: HashMap<String, Hook>,
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
struct JsonFilter {
pointer: String,
regex: String,
}
impl JsonFilter {
fn evaluate(&self, data: &serde_json::Value) -> Result<bool, WebhookeyError> {
trace!(
"Matching `{}` on `{}` from received json",
&self.regex,
&self.pointer,
);
let regex = Regex::new(&self.regex).map_err(WebhookeyError::Regex)?;
if let Some(value) = data.pointer(&self.pointer) {
let value = get_string(value)?;
if regex.is_match(&value) {
debug!("Regex `{}` for `{}` matches", &self.regex, &self.pointer);
return Ok(true);
}
}
debug!(
"Regex `{}` for `{}` does not match",
&self.regex, &self.pointer
);
Ok(false)
}
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields, rename_all = "lowercase")]
enum FilterType {
And(Vec<FilterType>),
Or(Vec<FilterType>),
#[serde(rename = "json")]
JsonFilter(JsonFilter),
}
impl FilterType {
fn evaluate(
&self,
request: &Request,
data: &serde_json::Value,
) -> Result<bool, WebhookeyError> {
match self {
FilterType::And(filters) => {
let (results, errors): (Vec<_>, Vec<_>) = filters
.iter()
.map(|filter| filter.evaluate(request, data))
.partition(Result::is_ok);
if errors.is_empty() {
Ok(results.iter().all(|r| *r.as_ref().unwrap())) // should never fail
} else {
errors.iter().for_each(|e| {
error!("Could not evaluate Filter: {}", e.as_ref().unwrap_err())
});
Err(WebhookeyError::InvalidFilter)
}
}
FilterType::Or(filters) => {
let (results, errors): (Vec<_>, Vec<_>) = filters
.iter()
.map(|filter| filter.evaluate(request, data))
.partition(Result::is_ok);
if errors.is_empty() {
Ok(results.iter().any(|r| *r.as_ref().unwrap())) // should never fail
} else {
errors.iter().for_each(|e| {
error!("Could not evaluate Filter: {}", e.as_ref().unwrap_err())
});
Err(WebhookeyError::InvalidFilter)
}
}
FilterType::JsonFilter(filter) => filter.evaluate(data),
}
}
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
struct Hook {
@ -78,14 +167,7 @@ struct Hook {
signature: String,
ip_filter: Option<IpFilter>,
secrets: Vec<String>,
filters: HashMap<String, Filter>,
}
#[derive(Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
struct Filter {
pointer: String,
regex: String,
filter: FilterType,
}
#[derive(Debug, Error)]
@ -96,10 +178,16 @@ enum WebhookeyError {
Unauthorized(IpAddr),
#[error("Unmatched hook from `{0}`")]
UnmatchedHook(IpAddr),
#[error("Could not find field refered to in parameter `{0}`")]
InvalidParameterPointer(String),
#[error("Could not evaluate filter request")]
InvalidFilter,
#[error("IO Error")]
Io(std::io::Error),
#[error("Serde Error")]
Serde(serde_json::Error),
#[error("Regex Error")]
Regex(regex::Error),
}
#[derive(Debug)]
@ -198,7 +286,7 @@ fn replace_parameters(
Ok(result.join(""))
}
fn get_string(value: &serde_json::Value) -> Result<String> {
fn get_string(value: &serde_json::Value) -> Result<String, WebhookeyError> {
match &value {
serde_json::Value::Bool(bool) => Ok(bool.to_string()),
serde_json::Value::Number(number) => Ok(number.to_string()),
@ -210,50 +298,29 @@ fn get_string(value: &serde_json::Value) -> Result<String> {
}
}
fn filter_match(
fn get_command(
hook_name: &str,
hook: &Hook,
filter_name: &str,
filter: &Filter,
request: &Request,
data: &mut serde_json::Value,
) -> Result<Option<String>> {
trace!("Matching filter `{}` of hook `{}`", filter_name, hook_name);
let regex = Regex::new(&filter.regex)?;
) -> Result<String> {
trace!("Replacing parameters for command of hook `{}`", hook_name);
for parameter in get_parameter(&hook.command)? {
let parameter = parameter.trim();
if let Some(json_value) = data.pointer(parameter) {
*data.pointer_mut(parameter).unwrap() =
*data
.pointer_mut(parameter)
.ok_or_else(|| WebhookeyError::InvalidParameterPointer(parameter.to_string()))? =
serde_json::Value::String(get_string(json_value)?);
}
}
if let Some(value) = data.pointer(&filter.pointer) {
let value = get_string(value)?;
if regex.is_match(&value) {
debug!("Filter `{}` of hook `{}` matched", filter_name, hook_name);
return Ok(Some(replace_parameters(
&hook.command,
&request.headers(),
data,
)?));
}
}
debug!(
"Filter `{}` of hook `{}` did not match",
filter_name, hook_name
);
Ok(None)
replace_parameters(&hook.command, &request.headers(), data)
}
fn execute_hooks(request: &Request, data: Data) -> Result<Hooks, WebhookeyError> {
fn get_commands(request: &Request, data: Data) -> Result<Hooks, WebhookeyError> {
let mut buffer = Vec::new();
let size = data
.open()
@ -277,7 +344,7 @@ fn execute_hooks(request: &Request, data: Data) -> Result<Hooks, WebhookeyError>
let signature = request
.headers()
.get_one(&hook.signature)
.ok_or_else(|| WebhookeyError::InvalidSignature)?;
.ok_or(WebhookeyError::InvalidSignature)?;
let secrets = hook
.secrets
@ -294,21 +361,17 @@ fn execute_hooks(request: &Request, data: Data) -> Result<Hooks, WebhookeyError>
let mut data: serde_json::Value =
serde_json::from_slice(&buffer).map_err(WebhookeyError::Serde)?;
for (filter_name, filter) in &hook.filters {
match filter_match(
&hook_name,
&hook,
&filter_name,
&filter,
&request,
&mut data,
) {
Ok(Some(command)) => {
match hook.filter.evaluate(request, &data) {
Ok(true) => match get_command(&hook_name, &hook, &request, &mut data) {
Ok(command) => {
result.insert(hook_name.to_string(), command);
break;
}
Ok(None) => {}
Err(e) => error!("{}", e),
},
Ok(false) => info!("Filter for `{}` did not match", &hook_name),
Err(error) => {
error!("Could not match filter for `{}`: {}", &hook_name, error)
}
}
}
@ -328,7 +391,7 @@ impl FromDataSimple for Hooks {
type Error = WebhookeyError;
fn from_data(request: &Request, data: Data) -> data::Outcome<Self, Self::Error> {
match execute_hooks(&request, data) {
match get_commands(&request, data) {
Ok(hooks) => {
if hooks.inner.is_empty() {
let client_ip = &request
@ -357,7 +420,7 @@ impl FromDataSimple for Hooks {
fn receive_hook<'a>(address: SocketAddr, hooks: Hooks) -> Result<Response<'a>> {
info!("Post request received from: {}", address);
for (name, command) in hooks.inner {
hooks.inner.iter().for_each(|(name, command)| {
info!("Execute `{}` from hook `{}`", &command, &name);
match run_script::run(&command, &vec![], &ScriptOptions::new()) {
@ -370,15 +433,15 @@ fn receive_hook<'a>(address: SocketAddr, hooks: Hooks) -> Result<Response<'a>> {
error!("Execution of `{}` failed: {}", &command, e);
}
}
}
});
Ok(Response::new())
}
fn get_config() -> Result<File> {
// Look for systemwide config..
if let Ok(config) = File::open("/etc/webhookey/config.yml") {
info!("Loading configuration from `/etc/webhookey/config.yml`");
// Look for config in CWD..
if let Ok(config) = File::open("config.yml") {
info!("Loading configuration from `./config.yml`");
return Ok(config);
}
@ -397,9 +460,9 @@ fn get_config() -> Result<File> {
}
}
// ..look for config in CWD..
if let Ok(config) = File::open("config.yml") {
info!("Loading configuration from `./config.yml`");
// ..look for systemwide config..
if let Ok(config) = File::open("/etc/webhookey/config.yml") {
info!("Loading configuration from `/etc/webhookey/config.yml`");
return Ok(config);
}
@ -437,6 +500,7 @@ mod tests {
#[test]
fn secret() {
let mut hooks = HashMap::new();
hooks.insert(
"test_hook".to_string(),
Hook {
@ -444,9 +508,13 @@ mod tests {
signature: "X-Gitea-Signature".to_string(),
ip_filter: None,
secrets: vec!["valid".to_string()],
filters: HashMap::new(),
filter: FilterType::JsonFilter(JsonFilter {
pointer: "*".to_string(),
regex: "*".to_string(),
}),
},
);
let config = Config { hooks: hooks };
let rocket = rocket::ignite()