---
title: "Event data selection"
author: "Gert Janssenswillen"
date: 2/12/2015
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Event data selection}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---
```{r echo = F, message=F}
library(edeaR)
library(ggplot2)
library(dplyr)
```

The goal of this vignette is to illustrated the different methods for performing data selection provided by the package `edeaR`. The data from municipality 1 of the BPI challenge 2015 will be used as running example. A preprocessed event log is available in the package under te name `BPIC15_1`.

```{r}
data(BPIC15_1)
```

### Select on activity frequency

In the vignette discussing the descriptives, it was already observed than only a limited fraction of the activities occurs very often. Below, we repeat the cumulative distribution of the activity frequences.

```{r echo = F, fig.width = 7}
activity_information <- activities(BPIC15_1)
ggplot(activity_information) +
	stat_ecdf(aes(absolute_frequency), lwd = 1, col = "#0072B2") + 
	scale_x_continuous(breaks = seq(0, 1000, by = 100)) + 
	xlab("Absolute activity frequencies") +
	ylab("Cumulative percentage") +
	geom_hline(aes(yintercept = 0.75))
```

A horizontal line has been added at 75\%. Recall that 75% of the activities only occures less than 100 times, approximately. To select only these activities, we use the filter `activity_frequency`, as follows

```{r}
filtered_log <- filter_activity_frequency(BPIC15_1, percentile_cut_off = 0.25, reverse = T)
activities(filtered_log) %>% select(absolute_frequency) %>% summary
```

Note that the combination of a percentile cut off of 25\% and reverse equal to TRUE, will select __all but__ the 25\% most frequenct activities, i.e. the 75% least frequent activities. It can be seen that the remaining activities have a absolute frequency of 105 or less.

### Select on throughput time

The throughput time of the original eventlog is visualized in the graph below. It can be observed that most cases have a throughput time lower that circa 100 days, while there are some outliers.

```{r fig.width = 7}
case_throughput <- throughput_time(BPIC15_1, "case")
ggplot(case_throughput) +
	geom_histogram(aes(throughput_time), fill = "#0072B2", binwidth = 10) +
	xlab("Duration (in days)") +
	ylab("Number of cases")
```

To discard the outliers with a throughput time greater than 500 days, we can use the filter `throughput_time` as follows. 

```{r fig.width = 7}
filtered_log <- filter_throughput_time(BPIC15_1, lower_threshold = 0, upper_threshold = 500)
case_throughput <- throughput_time(filtered_log, "case")
ggplot(case_throughput) +
	geom_histogram(aes(throughput_time), fill = "#0072B2", binwidth = 10) +
	xlab("Duration (in days)") +
	ylab("Number of cases")
```

Alternatively, we could only look at the outliers and select for instance the 1\% longest cases. 

```{r fig.width = 7}
filtered_log <- filter_throughput_time(BPIC15_1, percentile_cut_off = 0.99, reverse = T)
throughput_time(filtered_log, "case")
```

### Select on time period

Finally, let us select cases on a specific time period. To get an idea of the distribution of cases over time, the graph below shows the number of cases according to the starting timestamp and complete timestamp

```{r  fig.width = 7}
start <- BPIC15_1 %>% group_by(case_concept.name) %>% summarize(timestamp = min(event_time.timestamp)) %>% mutate(type = "start")
complete <- BPIC15_1 %>% group_by(case_concept.name) %>% summarize(timestamp = max(event_time.timestamp)) %>% mutate(type = "end")
bind_rows(start, complete) %>% 
	ggplot() +
	geom_histogram(aes(timestamp, fill = type), binwidth = 60*60*24*30) +
	facet_grid(type ~ .) +
	scale_fill_brewer(palette = "Dark2") +
	theme(legend.position = "none")
```

### Contained

Suppose we are only interested in the first quarter of the year 2012. We can filter the data in different ways. Firstly, we can filter on cases which started and completed in this period.

```{r fig.width = 7}
library(lubridate)
a <- ymd_hms("20120101 00:00:00")
b <- ymd_hms("20120331 00:00:00")
filtered_log <- filter_time_period(BPIC15_1, a, b, "contained")
start <- filtered_log %>% group_by(case_concept.name) %>% summarize(timestamp = min(event_time.timestamp)) %>% mutate(type = "start")
complete <- filtered_log %>% group_by(case_concept.name) %>% summarize(timestamp = max(event_time.timestamp)) %>% mutate(type = "end")
bind_rows(start, complete) %>% 	
	ggplot() +
	geom_histogram(aes(timestamp, fill = type), binwidth = 60*60*24*7) +
	facet_grid(type ~ .) +
	scale_fill_brewer(palette = "Dark2") +
	theme(legend.position = "none")
```

#### Started

Alternatively, we could select cases who started or completed in this period, respectively.

```{r fig.width = 7}
filtered_log <- filter_time_period(BPIC15_1, a, b, "start")
start <- filtered_log %>% group_by(case_concept.name) %>% summarize(timestamp = min(event_time.timestamp)) %>% mutate(type = "start")
complete <- filtered_log %>% group_by(case_concept.name) %>% summarize(timestamp = max(event_time.timestamp)) %>% mutate(type = "end")
bind_rows(start, complete) %>% 		ggplot() +
	geom_histogram(aes(timestamp, fill = type), binwidth = 60*60*24*7) +
	facet_grid(type ~ .) +
	scale_fill_brewer(palette = "Dark2") +
	theme(legend.position = "none")
```

#### Completed
```{r fig.width = 7}
filtered_log <- filter_time_period(BPIC15_1, a, b, "complete")
start <- filtered_log %>% group_by(case_concept.name) %>% summarize(timestamp = min(event_time.timestamp)) %>% mutate(type = "start")
complete <- filtered_log %>% group_by(case_concept.name) %>% summarize(timestamp = max(event_time.timestamp)) %>% mutate(type = "end")
bind_rows(start, complete) %>% 	
	ggplot() +
	geom_histogram(aes(timestamp, fill = type), binwidth = 60*60*24*7) +
	facet_grid(type ~ .) +
	scale_fill_brewer(palette = "Dark2") +
	theme(legend.position = "none")
```


#### Intersected

Still another option is to select cases who intersected the time period, i.e. at least part of the case happened in the time period.

```{r fig.width = 7}
filtered_log <- filter_time_period(BPIC15_1, a, b, "intersecting")
start <- filtered_log %>% group_by(case_concept.name) %>% summarize(timestamp = min(event_time.timestamp)) %>% mutate(type = "start")
complete <- filtered_log %>% group_by(case_concept.name) %>% summarize(timestamp = max(event_time.timestamp)) %>% mutate(type = "end")
bind_rows(start, complete) %>%
	ggplot() +
	geom_histogram(aes(timestamp, fill = type), binwidth = 60*60*24*7) +
	facet_grid(type ~ .) +
	scale_fill_brewer(palette = "Dark2") +
	theme(legend.position = "none")
```

#### Trim

Finally, we can trim the cases to the time period.

```{r eval = F, fig.width = 7}
filtered_log <- filter_time_period(BPIC15_1, a, b, "trim")
start <- filtered_log %>% group_by(case_concept.name) %>% summarize(timestamp = min(event_time.timestamp)) %>% mutate(type = "start")
complete <- filtered_log %>% group_by(case_concept.name) %>% summarize(timestamp = max(event_time.timestamp)) %>% mutate(type = "end")
bind_rows(start, complete) %>%
	ggplot() +
	geom_histogram(aes(timestamp, fill = type), binwidth = 60*60*24*7) +
	facet_grid(type ~ .) +
	scale_fill_brewer(palette = "Dark2") +
	theme(legend.position = "none")
```

## Other filters

Other filters provided are listed below. Look at the help file for their workings.

*	Endpoint filter
*	Precendence filter
*	Trace frequency filter
*	Trace length filter
*	Trim filter




