Mural API - Retrieving all sticky notes from a Mural Canvas

HCross3

My team holds a series of canvas's which have been used for our Innovation Events, these canvas's often hold upwards of 200-300 sticky notes. Recently I've been attempting to use the new Mural API to pull the sticky notes from a Mural with the intent of analysis.

I've been following the tutorial guide and code as provided on a range of test and real canvas's, from this i seem to be running into a problem where i am only able to pull 99 widgets from a canvas. A look at the Mural API documentation seems to indicate that the widgets are pulled in sets of 100 and paginated.

Is there any way to pull all of the widgets as a single page? When i attempted to increase the limit it throws an error when above 100. If not then can someone please help with understanding how to page through the widget pages in order to combine into a single dataframe within python.

spackows

Hi, @HCross3 .

I created a sample Python notebook that paginates through widgets results.

It's number 8 in this list:

https://github.com/spackows/MURAL-API-Samples

I don't put the results in a DataFrame in that example, but it's easy to do like this:

import pandas as pd
widgets_df_full = pd.DataFrame( widgets_arr )
widgets_df

That's assuming you have an array of widget JSON structures, like what is returned from the listAllWidgets function in the sample notebook above.

Also, I often only want to work with a subset of the data that's returned, so I also do something like this:

widgets_df = widgets_df_full[["id","x", "y","width","height","text","type","shape","style"]]
widgets_df

Let me know if you have any trouble or if you have suggested improvements for the samples. 🙂

HCross3

Hello,

Thanks for the guidance on the above, its given me a direction to work toward.

When attempting to use solution 8 provided in the Mural API samples i receive the below error. It seems like there is something within my test Mural (Which is a copy of a real workshop event canvas) that when attempting to pull as a widget does not have a "text" value. Are there any type of widgets which need to be filtered out prior to that segment which may be causing the error?

I then fixed that issue by filtering only for sticky notes prior to checking for a "text" value as below. As the purpose of this code is to pull the information from the sticky notes for analysis this does not impact my use.

The main issue and cause for this post is the following error i receive. This error seems to be around accessing multiple pages of widgets as while i can use the provided example code to search for them it seems to be unable to find/access the new page.

If possible @spackows can you please provide some guidance on this issue?

Thanks, Haris.

spackows

Hi, @HCross3

You're right about some widgets not having a text field. Your solution of filtering on widget type is good.

Another option would be to check something like:

if "text" in widget

That way, if someone put their comments in a text box (maybe a shape can have text too) instead of a sticky note, you would still collect their comment.

With respect to the "path not found on our servers" error, that's new for me. I haven't encountered that error before.

@Anuj , if you see this message, what is the process for reporting problems like that path-not-found error described above?

HCross3

Hi @spackows

Thanks for the help so far. Since my last post I've found that the error that prevented me from iterating through the pages which was due to a change I had made based off of incorrect assumptions in the code.

In the sample provided you've limited it too 3 widgets per page to show the use of the page feature which i modified as i needed to use this on real data. As a result of removing the limit on widgets the following if statement would always throw an error due to incorrect syntax. This was due to the code that added the search for the next page feature was dependant on the limiter feature being implemented.

The changes/fix i implemented was to remove the limiter and modify line 5 to assume the "next" syntax was the first to be added as opposed to the limiter. In hindsight this was an small error caused by unfamiliarity with python etc but i've put a screenshot below to show for future reference.

Anuj

Hi @HCross3: Thanks for checking out the MURAL API and building cool stuff using it! Per your last post, it seems like you were able to make progress and get past the issue you were facing earlier. Thank you @spackows for your help and guidance!