Using TaskPipes versus Writing Code

From our point of view, one of the most exciting aspects of TaskPipes is the fact that it allows non-developers to do things that previously only developers could do.

Need to extract data straight from a PDF, modify it, and export it into a database, every time you get a message from a certain email address?

Or perhaps you need to pull data from a webpage, use it in an intermediate calculation, and export it to your accounting software?

Each of these processes by themselves would take a developer a dozen hours of coding and testing to complete.

With TaskPipes, even non-technical users can get set up in just a few minutes.

An Example

Let’s look at an example of how you would approach a typical task in code versus with TaskPipes. Maybe you have a list of weekly usage for each customer in an Excel file:

Screen Shot 2015-10-05 at 20.39.39

Now, we want to get the data so that we have a single line entry for each user for every week. In TaskPipes, this is simple with the “Columns to Rows” method.

Screen Shot 2015-10-05 at 20.42.32

We say that we want to perform this action from the third column onwards, would like to “pull down” only one column in every chunk of data, and would like to also pull down the column headers (the week labels).

We are then left with the data in this format:

Screen Shot 2015-10-05 at 20.45.35

We now need to perform some simple data manipulation: pulling out the week number, and renaming a header:

Screen Shot 2015-10-06 at 11.44.47

From there, we apply a filter to only allow through non-zero values by applying a filter, and enter our database credentials to load these transactions into a MySQL database.

This leaves us with the following process:

Screen Shot 2015-10-05 at 20.49.39

…and the following data:

Screen Shot 2015-10-05 at 20.50.04

If we were to automate this process with code, it might look something like this:

def run():
    # Step 1 - "Choose Input Data"
    rows = []
    path = BASE_PATH + '/using_taskpipes_vs_code.xlsx'
    workbook = open_workbook(path)
    worksheet = workbook.sheet_by_index(0)
    for r in range(worksheet.nrows):
        row = []
        for c in range(worksheet.ncols):
            cell_value = get_and_cast_value_in_cell(worksheet, r, c)

    headers = rows.pop(0)

    # Step 2 - "Columns to Rows"
    new_rows = []
    first_column_to_split = 2
    headers_to_split = headers[first_column_to_split:]
    for row in rows:
        cells_to_copy = row[:first_column_to_split]
        cells_to_split = row[first_column_to_split:]

        for cell, header in zip(cells_to_split, headers_to_split):
            new_rows.append(cells_to_copy + [header] + [cell])

    headers = headers[:first_column_to_split] + ['', '']
    rows = new_rows

    # Step 3 - "Extract Text"
    column_index = 2
    headers.append('Week Number')
    for row in rows:
        cell = row[column_index]
        start_index = cell.index('Week') + 4
        extracted_text = cell[start_index:]

    # Step 4 - "Remove Column"
    column_index = 2
    del headers[column_index]
    for row in rows:
        del row[column_index]

    # Step 5 - "Rename Column"
    headers[2] = 'Transaction Amount ($)'

    # Step 6 - "Filter"
    new_rows = []
    for row in rows:
        if row[0] != 0:
    rows = new_rows

    # Step 7 - "Move Column"
    old_index = 3
    new_index = 2
    headers.insert(new_index, headers.pop(old_index))
    for row in rows:
        row.insert(new_index, row.pop(old_index))

    # Step 8 - "Insert into MySQL Database"
    database = connect_to_database(
    table = database.table('table_name')

    for row in rows:
        entry = create_entry(table, **dict(zip(headers, row)))

Reviewing this code, it is clear that this has been built with the most basic functionality. It doesn’t cover any of the edge-cases that so often appear when dealing with data like this. Furthermore, this would be fairly tricky to debug.

There are a few key reasons why we think TaskPipes tackles this better than code:

  1. Accessibility to non-technical users. Obviously code was never designed to be approachable to non-technical users. TaskPipes was.
  2. Easy To Modify. But what if these incoming Excel files change format? Using code, you’d have to change all the list indices etc. yourself or, even worse, lean on your friend in the software department to help you out. With TaskPipes, it is as simple as changing a couple of drop-downs.
  3. Visual Ease of Use. The user interface of TaskPipes makes it simple to see how a process is working, and being able to see what the data looks like at each step makes it straightforward to ensure that the process is working as expected. To do something similar in code, you’d have to include a load of breakpoints.

See what processes you can automate by heading over to now.

Using TaskPipes versus Writing Code

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s