The Last Month in Evaluation (with some thoughts on AEA)

Posted by

It’s been a minute since we took a look at the evaluation literature. There’s a lot of stuff that gets published with no way to comb through it all. Click below to go to PubTrawlr’s recap of the last 30 days of some of the main Evaluation journals.

Like it? We plan to offer 30 day recaps as a preview of our larger and more comprehensive 101 Days of Science service. Be sure to sign up below and visit PubTrawlr for more information.

BUT THAT’S NOT ALL

Dr. Victoria Scott and I just submitted a manuscript where we sent a bot to AEA’s 2020 conference. We scraped all the title and abstracts to try to get a sense of what the core themes and topics were.

The figure below show the phrases and words that occurred most often in the abstracts. That good ole’ standby “lessons learned” make a central appearance.

One of my favorite consequences of lemmatization is that “data” becomes the little-used “datum”

We also took things a step further and did some extractive summarization on the topic cluster. Extractive (as opposed to Abstractive) Summarization finds the most representative sentences by measuring their lexical similarity to one another. The figure below shows an excerpt from the table where we report out these summaries. Extractive Summarization can get a bit clunky, but it generally does a good job at getting the core themes across.

Something else we saw was that that the conference theme “Shine a Light” was all over the title submissions. Because we didn’t have access all the submissions, there was no way to tell whether being so explicit increases the likelihood of being accepted. It would be an interesting secondary analysis, though. We’re actually doing something really similar with our RFP project

Also, as a big Wolf Parade fan, when I read, “Shine a Light,” my mind always goes to this banger. 2005 was awesome year.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s