Search:
Sort by:		Search Reset

Dr. Hai-Jew discusses her research on coding imagery and multimedia.

Do Multimedia and Imagery Mean More than Meets the Eye?

By Taylor Chernisky on Aug 30, 2017

Every second, your brain processes 400 billion bits of information. This information can be words that you see or hear, graphics, or images. In order to process this information, your brain must be able to quickly analyze the information and determine its relevance to you, and then either code it into your memory or discard it. Considering that we are only aware of 2,000 of these bits of information per second, it is safe to say that our brains do more discarding than coding.

Dr. Shalin Hai-Jew, editor of multiple IGI Global books, has recently authored a new publication, Techniques for Coding Imagery and Multimedia: Emerging Research and Opportunities, that looks at how images and multimedia information can be coded in order to be properly understood and categorized. She recently talked with IGI Global about the scope of this research, as well as some future opportunities with this research and why it is important to study.

Q: What does it mean to “code” imagery and multimedia?

Coding here, generally, refers to the extraction of meaning from the imagery and multimedia by analyzing the works and finding patterns. Coding work may also involve the categorizing and labeling of the respective works. In the same way that people encode meanings in writing, they also encode meanings in their imagery and in multimedia objects. While coding itself is already very difficult and complex in the analysis of texts, the high dimensionality of imagery and multimodal and mixed forms of media adds that much more complexity and difficulty. Those who use an emergent, bottom-up coding approach with such digital contents will have to contend with a variety of messages and dimensions, often with very few computational supports for their coding work. Some may argue that coding is also about co-constructing meaning from the digital objects as “prompts,” without the implication of inherent built-in meanings to the respective works. And of course, it is important to document and define these efforts for the Methods sections of papers and to support others’ research works, both contemporaneously and into the future.

It strikes me that there are lots of right ways to do this—to code image and multimedia “data,” if you will. I think that it is okay to explore and experiment, to go with some initial naïve approaches, to be banal, even, in coding, so as to lay the groundwork for more nuanced and sensitive work later on. To generalize, though, if done manually, this is painstaking and iterative work. It really helps to have a system and a plan that is informed by both a priori assumptions as well as the image and multimedia sets.

Q: How do you source imagery for research? Likewise, how do you source multimedia for research?

There were a number of sources for the digital research materials in this book, including the Web, Wikipedia, article collections from a media organization, Instagram, and others. It’s pretty remarkable how many web browser add-ons, as well as software add-ons, are out there to enable the collection of data. If you tend towards “greedy” when it comes to data collection, as I tend to be, the technologies enable a pretty mass grab of digital stuff.

Much of what is out in the world today is multimedia. There are very few objects that are purely one thing or another (no pure texts without imagery, no slideshows without some audio overlay, no video without some HTML interactivity, etc.). There is a lot going on in combinatorial ways, with resulting combinatorial complexities. The engaging with multimedia opens up new ways of thinking, with integrated modalities to explore, express, and analyze.

Q: What is quality “coding” in relation to imagery? What is quality “coding” in relation to multimedia?

“Quality” coding depends on the actual research context and what the researchers are looking for and trying to study. There are generalizable points about quality. I think “quality” has something to do with coding accuracy and with coding methods (and how accepted these methods are in particular domains). There is something about coding efficiency that has to be achieved…and maybe something about reproducibility (to a degree). Ultimately, learned insights are important as well. You want to code towards relevance, not just pattern identification.

Coding Word Tree Visualization

Q: Could you tell us about Techniques for Coding Imagery and Multimedia: Emerging Research and Opportunities?

Sure, the book is organized in three sections: (1) Coding Imagery for Sensemaking, (2) Exploring Social Phenomena, and (3) Image and Multimedia Coding in Academia. It features seven chapters and a preface and an epilogue. Topically, there are explorations of a #selfie #humor image-set from Instagram, an exploration of snacks around the world, technology manifestos, American citizenship renunciation, and academic assessment in evaluating student-created imagery and multimedia. The topics range broadly because I needed to seed the explorations in order to collect data and to apply the coding techniques, but at heart, this work is about applied coding of imagery and multimedia.

Q: What are some necessary skills for coding imagery and multimedia? Are these skills easy to develop?

On the one hand, I want to argue that humans have an inherent capability for visual analysis, and they do. Yet, I also know that researchers are made, not born…and I know a lot of training goes into the work of research, coding, and analysis. So I’ll put myself somewhere between those two assertions, closer to the training end. That said, the technologies described in this work are all publicly available tools. The commercial software has high performance and functionality. Some of the technology is free and open-source. What is cutting-edge now may not really even manifest in some locales around the world until years and years later. What is faddish now may recur in later waves, in different forms, and in different geographies, and among different peoples. There are valuable insights from local contexts that are valuable even if the work is not bleeding edge.

Q: How did you get interested in this area of study?

In my work in instructional design, I am in online spaces a lot, and I end up doing a lot of different things. As is typical with book proposals, I saw a gap and thought it might be fun to see what was out there in terms of coding complex digital objects, like imagery and multimedia. I thought, “Wow, I’ll bet a lot of people are coding imagery and multimedia as part of their research, and I’ll bet they’ll be willing to share some techniques!” This was back in 2015. It turned out that while I think the first part of my hunch was right, the latter part was not, thus, an edited book turned into an authored one. I would have preferred having a wide stable of authors for a range of insights, but I think there is still a lot of diversity in this work.

I’m all about going for the best that you can achieve at a time, with what’s available, and then not regretting what could have been. In this time period, I also had a partial retinal detachment. During this book’s gestation, I spent part of that time recovering and learning to appreciate just how critical human vision can be for some types of coding and analysis.

I want to be ambitious in question-asking and not to stop myself just because a work seems hard or initially impossible.

Q: Why do you believe that this research is important in today’s world?

Having some efficient ways to code imagery and multimedia—with a combination of manual and computational methods—is important since so many researchers are exploring relevant research questions with high-dimensional data. While there’s a lot of value in studying small data sets and even singular examples, there are different types of insights attainable from large structured and semi-structured datasets. Researchers learn from each other’s methods and approaches all the time, and this is one area in which researchers’ methods can be high-impact and valuable to others. We’re in an age when there is so much raw digital data available out there, much of it consumed locally only, that we need more effective methods to harness this. In the same way that there is a “great unread” in terms of books, there is a “great unread” of social shared digital data. There is a lot of niche content—stuff in the “long tail”—that is wholly unexplored. The little that automation enables exploration of is seen by those in corporations and government but not so much common researchers because of prohibitive costs…and the coarse scale of the analytics (often focused around public personalities and brands and politics).

Q: Is this research relevant to any trending topics in today’s news?

Actually, quite so! With so much talk about #fakenews, those in mass media and in information-based organizations are working hard to separate the wheat from the chaff in terms of public opinions. So much in a modern democracy rides on public awareness and public opinion. Public opinion polls are still very much with us and relevant, but so too are “human sensor networks” in terms of postings on social media platforms. While three are automated ways to extract meaning from texts, imagery, and video, at scale, those approaches are not low-cost and not particularly nuanced. For smaller sets of semi-structured data (like imagery and multimedia files), human-led analytics can be much more effective and complex and culture-informed. This work shows ways to get closer to #realnews and #realthoughts using informed, interpretive, and systematic analytical approaches to imagery and multimedia contents.

Q: What are some of the research opportunities in this area?

We need to find efficient ways to find meaning from digital contents from social media platforms. Based on coding imagery and multimedia, we need to find ways to answer particular research questions across all domains. We need ways to provide fair and efficient assessments of multimodal student work. We also need to harness various technologies to enable efficient coding of images and multimedia, without the fairly huge amounts of noise based on what is available today. (While IBM Watson’s API does enable access to some impressive machine coding, not all questions can be efficiently answered through that tool.)

I think this book can be a positive conversation starter, to see how people code imagery and multimedia where they are. My “local maxima” is not the “global maxima,” and there is a lot of room for us to learn from each other. Together, we can contribute to defining this space.

IGI Global would like to thank Dr. Hai-Jew for taking the time to discuss her recent publication with us. Please take a moment to review Dr. Hai-Jew’s publications below:

Newsroom Contact
Caroline Campbell
Promotions Assistant
ccampbell@igi-global.com
(717) 533-8845, ext. 144
www.igi-global.com/

Browse for more posts in:

Media and Communications

No comments Comments

Be the first to comment!

More from IGI Global

Apr 25, 2024 How AI Changes the Game with Targeted Advertising and Rapid Upscaling

How can advertisers possibly keep up with billions of unique individuals on the planet? Enter artificial intelligence.

IGI GlobalRead More

Business and Management Marketing Books & E-Books Research Trends

Apr 23, 2024 Participate in the IGI Global Annual Academic Publishing Trends & Open Access Survey 2024

In its second year, the IGI Global Annual Academic Publishing Trends & Open Access Survey 2024 seeks to create a realistic outlook on problems faced by the academic community and their potential solutions.

IGI GlobalRead More

Resources for Librarians Resources for Researchers Open Access

Apr 18, 2024 Uniting Nations with Multinational Electronic Health Records to Stop Lassa Fever's Pandemic Threat!

The World Health Organization (WHO) has reported a surge in Lassa fever cases in Nigeria, emphasizing the urgent need for containment measures.

IGI GlobalRead More

Medical, Healthcare, and Life Sciences Healthcare Information Systems Books & E-Books Research Trends

Apr 16, 2024 IGI Global Announces the 2024 Journal Reviewers Award Winners

IGI Global congratulates the winners of this year's Journal Reviewer Award

IGI GlobalRead More

Journals Awards & Recognition Open Access

Apr 15, 2024 What Defines Authorship and Who Should Be Included in a Manuscript’s Byline?

For decades, academic publishing has been plagued with discrepancies surrounding authorship of scholarly research...

IGI GlobalRead More

Books & E-Books Acquisitions

Apr 15, 2024 Doody’s Selects IGI Global Books for Special Topics List

Two IGI Global publications have been recognized by Doody's for their excellence and niche topic focus.

IGI GlobalRead More

Medical, Healthcare, and Life Sciences Media and Communications Books & E-Books Awards & Recognition

Apr 11, 2024 Technology is Making Disparities Worse

Digital Inclusion Week underscored the urgent need for a national digital equity plan in the US due to disparities in internet access and digital skills.

IGI GlobalRead More

Apr 8, 2024 Nearly 90% of IGI Global’s Frontlist Books Now Indexed in Scopus

The majority of IGI Global's books Frontlist is now indexed by Scopus. Learn what this prestigious recognition means for the publisher and the experts behind these books.

IGI GlobalRead More

Books & E-Books Awards & Recognition Reviews & Indexing