Support so called conjunction filters
This introduces thee so called conjunction filters and therefore restructures the configuration file. The most obvious changes from an users perspective are that the `filters` field was renamed to `filter` and can, from now on, only support a single filter at first level. Thats why now different filter types are implemented, please consult the readme for further information on their usage. To reflect the changes the readme file is updated as well as the example config file contained in this repository. This is related to #8
This commit is contained in:
parent
891a8a70ae
commit
43b7fd5625
3 changed files with 185 additions and 96 deletions
62
README.md
62
README.md
|
@ -1,6 +1,6 @@
|
|||
# Webhookey
|
||||
Webhookey is a webserver listening for requests as for example sent by
|
||||
gitea's webhooks. Further, Webhookey allows you to specifiy rules
|
||||
Webhookey is a web server listening for requests as for example sent by
|
||||
gitea's webhooks. Further, Webhookey allows you to specify rules
|
||||
which are matched against the data received to trigger certain
|
||||
actions.
|
||||
|
||||
|
@ -15,7 +15,7 @@ Further, for Rocket we need to have the nightly toolchain installed:
|
|||
```
|
||||
|
||||
### Build Webhookey
|
||||
The webhookey project can be built for development:
|
||||
The Webhookey project can be built for development:
|
||||
``` sh
|
||||
cargo b
|
||||
```
|
||||
|
@ -27,7 +27,7 @@ or for releasing:
|
|||
|
||||
### Install Webhookey
|
||||
When a Rust toolchain installed you can also install Webhookey
|
||||
directly without cloning it manualy:
|
||||
directly without cloning it manually:
|
||||
``` sh
|
||||
cargo install --git https://git.onders.org/finga/webhookey.git webhookey
|
||||
```
|
||||
|
@ -51,7 +51,7 @@ you built.
|
|||
Configuration syntax is YAML and it's paths as well as it's
|
||||
configuration format is described in the following sections.
|
||||
|
||||
### Configuration paths
|
||||
### Configuration Paths
|
||||
Following locations are checked for a configuration file:
|
||||
- `/etc/webhookey/config.yml`
|
||||
- `<config_dir>/webhookey/config.yml`
|
||||
|
@ -72,7 +72,7 @@ consists of the following fields:
|
|||
source addresses or ranges.
|
||||
- signature: Name of the HTTP header field containing the signature.
|
||||
- secrets: List of secrets.
|
||||
- filters: List of filters.
|
||||
- filter: Tree of filters.
|
||||
|
||||
Example:
|
||||
```yaml
|
||||
|
@ -87,10 +87,18 @@ hooks:
|
|||
secrets:
|
||||
- secret_key_01
|
||||
- secret_key_02
|
||||
filters:
|
||||
match_ref:
|
||||
pointer: /ref
|
||||
regex: refs/heads/master
|
||||
filter:
|
||||
or:
|
||||
- json:
|
||||
pointer: /ref
|
||||
regex: refs/heads/master
|
||||
- and:
|
||||
- json:
|
||||
pointer: /ref
|
||||
regex: refs/heads/a_branch
|
||||
- json:
|
||||
pointer: /after
|
||||
regex: f6e5fe4fe37df76629112d55cc210718b6a55e7e
|
||||
```
|
||||
|
||||
##### Command
|
||||
|
@ -137,17 +145,25 @@ Set the name of the HTTP header field containing the HMAC signature.
|
|||
Configure a list of secrets to validate the hook.
|
||||
|
||||
##### Filter
|
||||
Each filter must have following fields:
|
||||
- pointer: pointer to the JSON field according to [RFC
|
||||
6901](https://tools.ietf.org/html/rfc6901)
|
||||
- regex: regular expression which has to match the field pointed to by
|
||||
the pointer
|
||||
Filter can be either a concrete filter or a conjunction
|
||||
filter. Concrete filters return either true or false on specific
|
||||
constraints. Conjunction filters contain lists of filters which are
|
||||
evaluated and combined based on the type. The result is either used
|
||||
for parent conjunction filters or, if at the root, used to decide if a
|
||||
hook should be executed.
|
||||
|
||||
# TODOs
|
||||
## Use `clap` to parse command line arguments
|
||||
## Configure rocket via config.yml
|
||||
## Security
|
||||
### https support
|
||||
basically supported, but related to "Configure rocket via config.yml".
|
||||
### Authentication features
|
||||
## Use proptest or quickcheck for tests of parsers
|
||||
###### Conjunction Filters
|
||||
Conjunction filters contain lists of other filters.
|
||||
- `and`: Logical conjunction.
|
||||
- `or`: Logical disjunction.
|
||||
|
||||
###### Concrete Filters
|
||||
- `json`:
|
||||
|
||||
The `json` filter matches a regular expression on a field from the
|
||||
received JSON data.
|
||||
|
||||
- pointer: Pointer to the JSON field according to [RFC
|
||||
6901](https://tools.ietf.org/html/rfc6901).
|
||||
- regex: Regular expression which has to match the field pointed to
|
||||
by the pointer.
|
||||
|
|
31
config.yml
31
config.yml
|
@ -9,8 +9,8 @@ hooks:
|
|||
secrets:
|
||||
- secret_key_01
|
||||
- secret_key_02
|
||||
filters:
|
||||
match_ref:
|
||||
filter:
|
||||
json:
|
||||
pointer: /ref
|
||||
regex: refs/heads/master
|
||||
hook2:
|
||||
|
@ -22,19 +22,24 @@ hooks:
|
|||
secrets:
|
||||
- secret_key_01
|
||||
- secret_key_02
|
||||
filters:
|
||||
match_ref:
|
||||
pointer: /ref
|
||||
regex: refs/heads/master
|
||||
filter:
|
||||
and:
|
||||
- json:
|
||||
pointer: /ref
|
||||
regex: refs/heads/master
|
||||
- json:
|
||||
pointer: /after
|
||||
regex: f6e5fe4fe37df76629112d55cc210718b6a55e7e
|
||||
hook3:
|
||||
command: /usr/bin/local/script_xyz.sh
|
||||
signature: X-Gitea-Signature
|
||||
secrets:
|
||||
- secret_key03
|
||||
filters:
|
||||
match_ref:
|
||||
pointer: /ref
|
||||
regex: refs/heads/master
|
||||
match_after:
|
||||
pointer: /after
|
||||
regex: f6e5fe4fe37df76629112d55cc210718b6a55e7e
|
||||
filter:
|
||||
or:
|
||||
- json:
|
||||
pointer: /ref
|
||||
regex: refs/heads/master
|
||||
- json:
|
||||
pointer: /after
|
||||
regex: f6e5fe4fe37df76629112d55cc210718b6a55e7e
|
||||
|
|
188
src/main.rs
188
src/main.rs
|
@ -71,6 +71,95 @@ struct Config {
|
|||
hooks: HashMap<String, Hook>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize, Serialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
struct JsonFilter {
|
||||
pointer: String,
|
||||
regex: String,
|
||||
}
|
||||
|
||||
impl JsonFilter {
|
||||
fn evaluate(&self, data: &serde_json::Value) -> Result<bool, WebhookeyError> {
|
||||
trace!(
|
||||
"Matching `{}` on `{}` from received json",
|
||||
&self.regex,
|
||||
&self.pointer,
|
||||
);
|
||||
|
||||
let regex = Regex::new(&self.regex).map_err(WebhookeyError::Regex)?;
|
||||
|
||||
if let Some(value) = data.pointer(&self.pointer) {
|
||||
let value = get_string(value)?;
|
||||
|
||||
if regex.is_match(&value) {
|
||||
debug!("Regex `{}` for `{}` matches", &self.regex, &self.pointer);
|
||||
|
||||
return Ok(true);
|
||||
}
|
||||
}
|
||||
|
||||
debug!(
|
||||
"Regex `{}` for `{}` does not match",
|
||||
&self.regex, &self.pointer
|
||||
);
|
||||
|
||||
Ok(false)
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize, Serialize)]
|
||||
#[serde(deny_unknown_fields, rename_all = "lowercase")]
|
||||
enum FilterType {
|
||||
And(Vec<FilterType>),
|
||||
Or(Vec<FilterType>),
|
||||
#[serde(rename = "json")]
|
||||
JsonFilter(JsonFilter),
|
||||
}
|
||||
|
||||
impl FilterType {
|
||||
fn evaluate(
|
||||
&self,
|
||||
request: &Request,
|
||||
data: &serde_json::Value,
|
||||
) -> Result<bool, WebhookeyError> {
|
||||
match self {
|
||||
FilterType::And(filters) => {
|
||||
let (results, errors): (Vec<_>, Vec<_>) = filters
|
||||
.iter()
|
||||
.map(|filter| filter.evaluate(request, data))
|
||||
.partition(Result::is_ok);
|
||||
|
||||
if errors.is_empty() {
|
||||
Ok(results.iter().all(|r| *r.as_ref().unwrap())) // should never fail
|
||||
} else {
|
||||
errors.iter().for_each(|e| {
|
||||
error!("Could not evaluate Filter: {}", e.as_ref().unwrap_err())
|
||||
});
|
||||
|
||||
Err(WebhookeyError::InvalidFilter)
|
||||
}
|
||||
}
|
||||
FilterType::Or(filters) => {
|
||||
let (results, errors): (Vec<_>, Vec<_>) = filters
|
||||
.iter()
|
||||
.map(|filter| filter.evaluate(request, data))
|
||||
.partition(Result::is_ok);
|
||||
|
||||
if errors.is_empty() {
|
||||
Ok(results.iter().any(|r| *r.as_ref().unwrap())) // should never fail
|
||||
} else {
|
||||
errors.iter().for_each(|e| {
|
||||
error!("Could not evaluate Filter: {}", e.as_ref().unwrap_err())
|
||||
});
|
||||
|
||||
Err(WebhookeyError::InvalidFilter)
|
||||
}
|
||||
}
|
||||
FilterType::JsonFilter(filter) => filter.evaluate(data),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize, Serialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
struct Hook {
|
||||
|
@ -78,14 +167,7 @@ struct Hook {
|
|||
signature: String,
|
||||
ip_filter: Option<IpFilter>,
|
||||
secrets: Vec<String>,
|
||||
filters: HashMap<String, Filter>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize, Serialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
struct Filter {
|
||||
pointer: String,
|
||||
regex: String,
|
||||
filter: FilterType,
|
||||
}
|
||||
|
||||
#[derive(Debug, Error)]
|
||||
|
@ -96,10 +178,16 @@ enum WebhookeyError {
|
|||
Unauthorized(IpAddr),
|
||||
#[error("Unmatched hook from `{0}`")]
|
||||
UnmatchedHook(IpAddr),
|
||||
#[error("Could not find field refered to in parameter `{0}`")]
|
||||
InvalidParameterPointer(String),
|
||||
#[error("Could not evaluate filter request")]
|
||||
InvalidFilter,
|
||||
#[error("IO Error")]
|
||||
Io(std::io::Error),
|
||||
#[error("Serde Error")]
|
||||
Serde(serde_json::Error),
|
||||
#[error("Regex Error")]
|
||||
Regex(regex::Error),
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
|
@ -198,7 +286,7 @@ fn replace_parameters(
|
|||
Ok(result.join(""))
|
||||
}
|
||||
|
||||
fn get_string(value: &serde_json::Value) -> Result<String> {
|
||||
fn get_string(value: &serde_json::Value) -> Result<String, WebhookeyError> {
|
||||
match &value {
|
||||
serde_json::Value::Bool(bool) => Ok(bool.to_string()),
|
||||
serde_json::Value::Number(number) => Ok(number.to_string()),
|
||||
|
@ -210,50 +298,29 @@ fn get_string(value: &serde_json::Value) -> Result<String> {
|
|||
}
|
||||
}
|
||||
|
||||
fn filter_match(
|
||||
fn get_command(
|
||||
hook_name: &str,
|
||||
hook: &Hook,
|
||||
filter_name: &str,
|
||||
filter: &Filter,
|
||||
request: &Request,
|
||||
data: &mut serde_json::Value,
|
||||
) -> Result<Option<String>> {
|
||||
trace!("Matching filter `{}` of hook `{}`", filter_name, hook_name);
|
||||
|
||||
let regex = Regex::new(&filter.regex)?;
|
||||
) -> Result<String> {
|
||||
trace!("Replacing parameters for command of hook `{}`", hook_name);
|
||||
|
||||
for parameter in get_parameter(&hook.command)? {
|
||||
let parameter = parameter.trim();
|
||||
|
||||
if let Some(json_value) = data.pointer(parameter) {
|
||||
*data.pointer_mut(parameter).unwrap() =
|
||||
*data
|
||||
.pointer_mut(parameter)
|
||||
.ok_or_else(|| WebhookeyError::InvalidParameterPointer(parameter.to_string()))? =
|
||||
serde_json::Value::String(get_string(json_value)?);
|
||||
}
|
||||
}
|
||||
|
||||
if let Some(value) = data.pointer(&filter.pointer) {
|
||||
let value = get_string(value)?;
|
||||
|
||||
if regex.is_match(&value) {
|
||||
debug!("Filter `{}` of hook `{}` matched", filter_name, hook_name);
|
||||
|
||||
return Ok(Some(replace_parameters(
|
||||
&hook.command,
|
||||
&request.headers(),
|
||||
data,
|
||||
)?));
|
||||
}
|
||||
}
|
||||
|
||||
debug!(
|
||||
"Filter `{}` of hook `{}` did not match",
|
||||
filter_name, hook_name
|
||||
);
|
||||
|
||||
Ok(None)
|
||||
replace_parameters(&hook.command, &request.headers(), data)
|
||||
}
|
||||
|
||||
fn execute_hooks(request: &Request, data: Data) -> Result<Hooks, WebhookeyError> {
|
||||
fn get_commands(request: &Request, data: Data) -> Result<Hooks, WebhookeyError> {
|
||||
let mut buffer = Vec::new();
|
||||
let size = data
|
||||
.open()
|
||||
|
@ -277,7 +344,7 @@ fn execute_hooks(request: &Request, data: Data) -> Result<Hooks, WebhookeyError>
|
|||
let signature = request
|
||||
.headers()
|
||||
.get_one(&hook.signature)
|
||||
.ok_or_else(|| WebhookeyError::InvalidSignature)?;
|
||||
.ok_or(WebhookeyError::InvalidSignature)?;
|
||||
|
||||
let secrets = hook
|
||||
.secrets
|
||||
|
@ -294,21 +361,17 @@ fn execute_hooks(request: &Request, data: Data) -> Result<Hooks, WebhookeyError>
|
|||
let mut data: serde_json::Value =
|
||||
serde_json::from_slice(&buffer).map_err(WebhookeyError::Serde)?;
|
||||
|
||||
for (filter_name, filter) in &hook.filters {
|
||||
match filter_match(
|
||||
&hook_name,
|
||||
&hook,
|
||||
&filter_name,
|
||||
&filter,
|
||||
&request,
|
||||
&mut data,
|
||||
) {
|
||||
Ok(Some(command)) => {
|
||||
match hook.filter.evaluate(request, &data) {
|
||||
Ok(true) => match get_command(&hook_name, &hook, &request, &mut data) {
|
||||
Ok(command) => {
|
||||
result.insert(hook_name.to_string(), command);
|
||||
break;
|
||||
}
|
||||
Ok(None) => {}
|
||||
Err(e) => error!("{}", e),
|
||||
},
|
||||
Ok(false) => info!("Filter for `{}` did not match", &hook_name),
|
||||
Err(error) => {
|
||||
error!("Could not match filter for `{}`: {}", &hook_name, error)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
@ -328,7 +391,7 @@ impl FromDataSimple for Hooks {
|
|||
type Error = WebhookeyError;
|
||||
|
||||
fn from_data(request: &Request, data: Data) -> data::Outcome<Self, Self::Error> {
|
||||
match execute_hooks(&request, data) {
|
||||
match get_commands(&request, data) {
|
||||
Ok(hooks) => {
|
||||
if hooks.inner.is_empty() {
|
||||
let client_ip = &request
|
||||
|
@ -357,7 +420,7 @@ impl FromDataSimple for Hooks {
|
|||
fn receive_hook<'a>(address: SocketAddr, hooks: Hooks) -> Result<Response<'a>> {
|
||||
info!("Post request received from: {}", address);
|
||||
|
||||
for (name, command) in hooks.inner {
|
||||
hooks.inner.iter().for_each(|(name, command)| {
|
||||
info!("Execute `{}` from hook `{}`", &command, &name);
|
||||
|
||||
match run_script::run(&command, &vec![], &ScriptOptions::new()) {
|
||||
|
@ -370,15 +433,15 @@ fn receive_hook<'a>(address: SocketAddr, hooks: Hooks) -> Result<Response<'a>> {
|
|||
error!("Execution of `{}` failed: {}", &command, e);
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
Ok(Response::new())
|
||||
}
|
||||
|
||||
fn get_config() -> Result<File> {
|
||||
// Look for systemwide config..
|
||||
if let Ok(config) = File::open("/etc/webhookey/config.yml") {
|
||||
info!("Loading configuration from `/etc/webhookey/config.yml`");
|
||||
// Look for config in CWD..
|
||||
if let Ok(config) = File::open("config.yml") {
|
||||
info!("Loading configuration from `./config.yml`");
|
||||
|
||||
return Ok(config);
|
||||
}
|
||||
|
@ -397,9 +460,9 @@ fn get_config() -> Result<File> {
|
|||
}
|
||||
}
|
||||
|
||||
// ..look for config in CWD..
|
||||
if let Ok(config) = File::open("config.yml") {
|
||||
info!("Loading configuration from `./config.yml`");
|
||||
// ..look for systemwide config..
|
||||
if let Ok(config) = File::open("/etc/webhookey/config.yml") {
|
||||
info!("Loading configuration from `/etc/webhookey/config.yml`");
|
||||
|
||||
return Ok(config);
|
||||
}
|
||||
|
@ -437,6 +500,7 @@ mod tests {
|
|||
#[test]
|
||||
fn secret() {
|
||||
let mut hooks = HashMap::new();
|
||||
|
||||
hooks.insert(
|
||||
"test_hook".to_string(),
|
||||
Hook {
|
||||
|
@ -444,9 +508,13 @@ mod tests {
|
|||
signature: "X-Gitea-Signature".to_string(),
|
||||
ip_filter: None,
|
||||
secrets: vec!["valid".to_string()],
|
||||
filters: HashMap::new(),
|
||||
filter: FilterType::JsonFilter(JsonFilter {
|
||||
pointer: "*".to_string(),
|
||||
regex: "*".to_string(),
|
||||
}),
|
||||
},
|
||||
);
|
||||
|
||||
let config = Config { hooks: hooks };
|
||||
|
||||
let rocket = rocket::ignite()
|
||||
|
|
Loading…
Reference in a new issue