Named Entity Recognition (NER)

Named Entity Recognition (NER) will help us computationally identify people, places, and things (of various kinds) in a text or collection of texts. It is useful for extracting key information from texts. You might use NER to identify the most frequently appearing characters in a novel or build a network of characters (related to network analysis), or you might use NER to identify the geographic locations mentioned in texts, a first step toward mapping the locations (related to spatial analysis).

# Install spaCy for NER and other Natural Language Processing (NLP) tasks
!pip install -U spacy

Requirement already satisfied: spacy in c:\users\colto\anaconda3\lib\site-packages (3.7.2)
Requirement already satisfied: jinja2 in c:\users\colto\anaconda3\lib\site-packages (from spacy) (2.11.3)
Requirement already satisfied: packaging>=20.0 in c:\users\colto\anaconda3\lib\site-packages (from spacy) (21.3)
Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in c:\users\colto\anaconda3\lib\site-packages (from spacy) (1.0.5)
Requirement already satisfied: weasel<0.4.0,>=0.1.0 in c:\users\colto\anaconda3\lib\site-packages (from spacy) (0.3.4)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in c:\users\colto\anaconda3\lib\site-packages (from spacy) (3.0.9)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in c:\users\colto\anaconda3\lib\site-packages (from spacy) (2.27.1)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in c:\users\colto\anaconda3\lib\site-packages (from spacy) (1.0.10)
Requirement already satisfied: srsly<3.0.0,>=2.4.3 in c:\users\colto\anaconda3\lib\site-packages (from spacy) (2.4.8)
Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in c:\users\colto\anaconda3\lib\site-packages (from spacy) (1.1.2)
Requirement already satisfied: pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4 in c:\users\colto\anaconda3\lib\site-packages (from spacy) (2.5.2)
Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in c:\users\colto\anaconda3\lib\site-packages (from spacy) (4.64.0)
Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in c:\users\colto\anaconda3\lib\site-packages (from spacy) (3.3.0)
Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in c:\users\colto\anaconda3\lib\site-packages (from spacy) (2.0.10)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in c:\users\colto\anaconda3\lib\site-packages (from spacy) (2.0.8)
Requirement already satisfied: thinc<8.3.0,>=8.1.8 in c:\users\colto\anaconda3\lib\site-packages (from spacy) (8.2.1)
Requirement already satisfied: numpy>=1.19.0 in c:\users\colto\anaconda3\lib\site-packages (from spacy) (1.21.5)
Requirement already satisfied: smart-open<7.0.0,>=5.2.1 in c:\users\colto\anaconda3\lib\site-packages (from spacy) (6.4.0)
Requirement already satisfied: typer<0.10.0,>=0.3.0 in c:\users\colto\anaconda3\lib\site-packages (from spacy) (0.9.0)
Requirement already satisfied: setuptools in c:\users\colto\anaconda3\lib\site-packages (from spacy) (61.2.0)
Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in c:\users\colto\anaconda3\lib\site-packages (from spacy) (3.0.12)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in c:\users\colto\anaconda3\lib\site-packages (from packaging>=20.0->spacy) (3.0.4)
Requirement already satisfied: typing-extensions>=4.6.1 in c:\users\colto\anaconda3\lib\site-packages (from pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4->spacy) (4.8.0)
Requirement already satisfied: pydantic-core==2.14.5 in c:\users\colto\anaconda3\lib\site-packages (from pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4->spacy) (2.14.5)
Requirement already satisfied: annotated-types>=0.4.0 in c:\users\colto\anaconda3\lib\site-packages (from pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4->spacy) (0.6.0)
Requirement already satisfied: charset-normalizer~=2.0.0 in c:\users\colto\anaconda3\lib\site-packages (from requests<3.0.0,>=2.13.0->spacy) (2.0.4)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\users\colto\anaconda3\lib\site-packages (from requests<3.0.0,>=2.13.0->spacy) (1.26.9)
Requirement already satisfied: idna<4,>=2.5 in c:\users\colto\anaconda3\lib\site-packages (from requests<3.0.0,>=2.13.0->spacy) (3.3)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\colto\anaconda3\lib\site-packages (from requests<3.0.0,>=2.13.0->spacy) (2021.10.8)
Requirement already satisfied: blis<0.8.0,>=0.7.8 in c:\users\colto\anaconda3\lib\site-packages (from thinc<8.3.0,>=8.1.8->spacy) (0.7.11)
Requirement already satisfied: confection<1.0.0,>=0.0.1 in c:\users\colto\anaconda3\lib\site-packages (from thinc<8.3.0,>=8.1.8->spacy) (0.1.4)
Requirement already satisfied: colorama in c:\users\colto\anaconda3\lib\site-packages (from tqdm<5.0.0,>=4.38.0->spacy) (0.4.6)
Requirement already satisfied: click<9.0.0,>=7.1.1 in c:\users\colto\anaconda3\lib\site-packages (from typer<0.10.0,>=0.3.0->spacy) (8.0.4)
Requirement already satisfied: cloudpathlib<0.17.0,>=0.7.0 in c:\users\colto\anaconda3\lib\site-packages (from weasel<0.4.0,>=0.1.0->spacy) (0.16.0)
Requirement already satisfied: MarkupSafe>=0.23 in c:\users\colto\anaconda3\lib\site-packages (from jinja2->spacy) (2.0.1)

spaCy relies on machine learning models that were trained on a large amount of carefully-labeled texts. The English-language spaCy model that we’re going to use was trained on an annotated corpus called "OntoNotes": 2 million+ words drawn from "news, broadcast, talk shows, weblogs, usenet newsgroups, and conversational telephone speech," which were meticulously tagged by a group of researchers and professionals for people’s names and places, for nouns and verbs, for subjects and objects, and much more.

import spacy
from spacy import displacy # visualization based on spaCy
from collections import Counter # counting the results
import pandas as pd # dealing with dataframe

# Download the English-language model
!python -m spacy download en_core_web_sm

Collecting en-core-web-sm==3.7.1
  Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.7.1/en_core_web_sm-3.7.1-py3-none-any.whl (12.8 MB)
Requirement already satisfied: spacy<3.8.0,>=3.7.2 in c:\users\colto\anaconda3\lib\site-packages (from en-core-web-sm==3.7.1) (3.7.2)
Requirement already satisfied: pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4 in c:\users\colto\anaconda3\lib\site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (2.5.2)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in c:\users\colto\anaconda3\lib\site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (3.0.9)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in c:\users\colto\anaconda3\lib\site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (2.0.8)
Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in c:\users\colto\anaconda3\lib\site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (3.3.0)
Requirement already satisfied: weasel<0.4.0,>=0.1.0 in c:\users\colto\anaconda3\lib\site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (0.3.4)
Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in c:\users\colto\anaconda3\lib\site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (2.0.10)
Requirement already satisfied: srsly<3.0.0,>=2.4.3 in c:\users\colto\anaconda3\lib\site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (2.4.8)
Requirement already satisfied: packaging>=20.0 in c:\users\colto\anaconda3\lib\site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (21.3)
Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in c:\users\colto\anaconda3\lib\site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (1.1.2)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in c:\users\colto\anaconda3\lib\site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (2.27.1)
Requirement already satisfied: thinc<8.3.0,>=8.1.8 in c:\users\colto\anaconda3\lib\site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (8.2.1)
Requirement already satisfied: numpy>=1.19.0 in c:\users\colto\anaconda3\lib\site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (1.21.5)
Requirement already satisfied: jinja2 in c:\users\colto\anaconda3\lib\site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (2.11.3)
Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in c:\users\colto\anaconda3\lib\site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (3.0.12)
Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in c:\users\colto\anaconda3\lib\site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (4.64.0)
Requirement already satisfied: setuptools in c:\users\colto\anaconda3\lib\site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (61.2.0)
Requirement already satisfied: smart-open<7.0.0,>=5.2.1 in c:\users\colto\anaconda3\lib\site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (6.4.0)
Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in c:\users\colto\anaconda3\lib\site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (1.0.5)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in c:\users\colto\anaconda3\lib\site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (1.0.10)
Requirement already satisfied: typer<0.10.0,>=0.3.0 in c:\users\colto\anaconda3\lib\site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (0.9.0)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in c:\users\colto\anaconda3\lib\site-packages (from packaging>=20.0->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (3.0.4)
Requirement already satisfied: pydantic-core==2.14.5 in c:\users\colto\anaconda3\lib\site-packages (from pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (2.14.5)
Requirement already satisfied: annotated-types>=0.4.0 in c:\users\colto\anaconda3\lib\site-packages (from pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (0.6.0)
Requirement already satisfied: typing-extensions>=4.6.1 in c:\users\colto\anaconda3\lib\site-packages (from pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (4.8.0)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\colto\anaconda3\lib\site-packages (from requests<3.0.0,>=2.13.0->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (2021.10.8)
Requirement already satisfied: idna<4,>=2.5 in c:\users\colto\anaconda3\lib\site-packages (from requests<3.0.0,>=2.13.0->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (3.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\users\colto\anaconda3\lib\site-packages (from requests<3.0.0,>=2.13.0->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (1.26.9)
Requirement already satisfied: charset-normalizer~=2.0.0 in c:\users\colto\anaconda3\lib\site-packages (from requests<3.0.0,>=2.13.0->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (2.0.4)
Requirement already satisfied: confection<1.0.0,>=0.0.1 in c:\users\colto\anaconda3\lib\site-packages (from thinc<8.3.0,>=8.1.8->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (0.1.4)
Requirement already satisfied: blis<0.8.0,>=0.7.8 in c:\users\colto\anaconda3\lib\site-packages (from thinc<8.3.0,>=8.1.8->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (0.7.11)
Requirement already satisfied: colorama in c:\users\colto\anaconda3\lib\site-packages (from tqdm<5.0.0,>=4.38.0->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (0.4.6)
Requirement already satisfied: click<9.0.0,>=7.1.1 in c:\users\colto\anaconda3\lib\site-packages (from typer<0.10.0,>=0.3.0->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (8.0.4)
Requirement already satisfied: cloudpathlib<0.17.0,>=0.7.0 in c:\users\colto\anaconda3\lib\site-packages (from weasel<0.4.0,>=0.1.0->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (0.16.0)
Requirement already satisfied: MarkupSafe>=0.23 in c:\users\colto\anaconda3\lib\site-packages (from jinja2->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (2.0.1)
[+] Download and installation successful
You can now load the package via spacy.load('en_core_web_sm')

# Load the English language model
import en_core_web_sm
nlp = en_core_web_sm.load()

# Load the dataset
movie = pd.read_csv('Data/movie_dialogue.csv')
movie

	mid	cid	cname	mname	gender	wordcount	year	genres	comedy	thriller	drama	romance	lines
0	m0	u0	BIANCA	10 things i hate about you	f	959	1999	['comedy', 'romance']	True	False	False	True	They do not! / I hope so. / Let's go. / Okay -...
1	m0	u2	CAMERON	10 things i hate about you	m	527	1999	['comedy', 'romance']	True	False	False	True	They do to! / She okay? / Wow / No / The "real...
2	m0	u4	JOEY	10 things i hate about you	m	278	1999	['comedy', 'romance']	True	False	False	True	Listen, I want to talk to you about the prom. ...
3	m0	u5	KAT	10 things i hate about you	f	1217	1999	['comedy', 'romance']	True	False	False	True	Perm? / It's just you. / What? To completely d...
4	m0	u6	MANDELLA	10 things i hate about you	f	157	1999	['comedy', 'romance']	True	False	False	True	William - he asked me to meet him here. / Have...
...	...	...	...	...	...	...	...	...	...	...	...	...	...
2964	m98	u1455	ELSA	indiana jones and the last crusade	f	289	1989	['action', 'adventure', 'thriller', 'action', ...	False	True	False	False	I can reach it. I can reach it... / It's ours,...
2965	m98	u1456	HENRY	indiana jones and the last crusade	m	729	1989	['action', 'adventure', 'thriller', 'action', ...	False	True	False	False	Got lost in his own museum, huh? / The Name of...
2966	m98	u1457	INDY	indiana jones and the last crusade	m	1436	1989	['action', 'adventure', 'thriller', 'action', ...	False	True	False	False	It's... a leap of faith. Oh, God. / I'm going ...
2967	m99	u1463	INDIANA	indiana jones and the temple of doom	m	1500	1984	['action', 'adventure']	False	False	False	False	Then she must have run out of the room and you...
2968	m99	u1468	WILLIE	indiana jones and the temple of doom	f	950	1984	['action', 'adventure']	False	False	False	False	It's some kind of cult! And they've got the sa...

2969 rows × 13 columns

# Let's see the lines of the 1st character in the 1st movie
print(movie['lines'][0])

They do not! / I hope so. / Let's go. / Okay -- you're gonna need to learn how to lie. / I'm kidding.  You know how sometimes you just become this "persona"?  And you don't know how to quit? / Like my fear of wearing pastels? / What good stuff? / Me.  This endless ...blonde babble. I'm like, boring myself. / do you listen to this crap? / Then Guillermo says, "If you go any lighter, you're gonna look like an extra on 90210." / But / Well, no... / I was? / Tons / You know Chastity? / Hi. / Who knows?  All I've ever heard her say is that she'd dip before dating a guy that smokes. / Lesbian?  No. I found a picture of Jared Leto in one of her drawers, so I'm pretty sure she's not harboring same-sex tendencies. / I really, really, really wanna go, but I can't.  Not unless my sister goes. / Eber's Deep Conditioner every two days. And I never, ever use a blowdryer without the diffuser attachment. / You're sweet. / I counted on you to help my cause. You and that thug are obviously failing. Aren't we ever going on our date? / Where? / How is our little Find the Wench A Date plan progressing? / Forget French. / I don't want to know how to say that though.  I want to know useful things. Like where the good stores are.  How much does champagne cost?  Stuff like Chat.  I have never in my life had to point out my head to someone. / C'esc ma tete. This is my head / Gosh, if only we could find Kat a boyfriend... / Unsolved mystery.  She used to be really popular when she started high school, then it was just like she got sick of it or something. / The thing is, Cameron -- I'm at the mercy of a particularly hideous breed of loser.  My sister.  I can't date until she does. / No, no, it's my fault -- we didn't have a proper introduction --- / You're asking me out.  That's so cute. What's your name again? / Not the hacking and gagging and spitting part.  Please. / Can we make this quick?  Roxanne Korrine and Andrew Barrett are having an incredibly horrendous public break- up on the quad.  Again. / I did. / I have to be home in twenty minutes. / Sometimes I wonder if the guys we're supposed to want to go out with are the ones we actually want to go out with, you know? / Combination.  I don't know -- I thought he'd be different.  More of a gentleman... / He practically proposed when he found out we had the same dermatologist. I mean. Dr. Bonchowski is great an all, but he's not exactly relevant party conversation. / Would you mind getting me a drink, Cameron? / Joey. / Where did he go?  He was just here. / You might wanna think about it / Did you change your hair? / You know the deal.  I can ' t go if Kat doesn't go -- / Hi, Joey. / Neat... / Queen Harry? / Hopefully. / Expensive? / Patrick -- is that- a. / Is that woman a complete fruit-loop or is it just me? / No! I just wanted / I just wanted -- / Let go! / You looked beautiful last night, you know. / I guess I'll never know, will I? / God, you're just like him! Just keep me locked away in the dark, so I can't experience anything for myself / I'm not stupid enough to repeat your mistakes. / No. you didn't!  If you really thought I could make my own decisions, you would've let me go out with him instead of helping Daddy hold me hostage. / Why didn't you tell me? / But / You did what? / As in... / But you hate Joey / Why? / What? / I wish I had that luxury. I'm the only sophomore that got asked to the prom and I can't go, because you won ' t. / Like you care. / I don't get you.  You act like you're too good for any of this, and then you go totally apeshit when you get here. / I really don't think I need any social advice from you right now. / You are so completely unbalanced. / Yeah, he's your freak friend Mandella's boyfriend.  I guess since I'm not allowed to go out, I should obsess over a dead guy, too. / Like I'm supposed to know what that even means. / Can't you forget for just one night that you're completely wretched? / Bogey Lowenstein's party is normal, but you're too busy listening to Bitches Who Need Prozac to know that. / You're ruining my life'  Because you won't be normal, I can't be normal. / I think you're a freak.  I think you do this to torture me.  And I think you suck. / Oh, I thought you might have a date  I don't know why I'm bothering to ask, but are you going to Bogey Lowenstein's party Saturday night? / Oh my God, does this mean you're becoming normal? / Can you at least start wearing a bra? / Nowhere... Hi, Daddy. / I have a date, Daddy.  And he ' s not a captain of oppression like some men we know. / Fine.  I see that I'm a prisoner in my own house.  I'm not a daughter. I'm a possession! / He's not a "hot rod".  Whatever that is. / No, but / Daddy, I want to discuss the prom with you. It's tomorrow night -- / Why? / Daddy, no! / It's just a party. Daddy. / Daddy, people expect me to be there! / It's just a party. Daddy, but I knew you'd forbid me to go since "Gloria Steinem" over there isn't going -- / If you must know, we were attempting to go to a small study group of friends. / Daddy, I -- / But she doesn't want to date. / But it's not fair -- she's a mutant, Daddy! / What if she never starts dating? / Now don't get upset. Daddy, but there's this boy... and I think he might ask...

# Visualize the name entity spaCy recognizes (note that the results are not perfect)
document = nlp(movie['lines'][0])
displacy.render(document, style="ent")

They do not! / I hope so. / Let's go. / Okay -- you're gonna need to learn how to lie. / I'm kidding. You know how sometimes you just become this "persona"? And you don't know how to quit? / Like my fear of wearing pastels? / What good stuff? / Me. This endless ...blonde babble. I'm like, boring myself. / do you listen to this crap? / Then Guillermo PERSON says, "If you go any lighter, you're gonna look like an extra on 90210 DATE ." / But / Well, no... / I was? / Tons / You know Chastity? / Hi. / Who knows? All I've ever heard her say is that she'd dip before dating a guy that smokes. / Lesbian? No. I found a picture of Jared Leto PERSON in one CARDINAL of her drawers, so I'm pretty sure she's not harboring same-sex tendencies. / I really, really, really wanna go, but I can't. Not unless my sister goes. / Eber PERSON 's Deep Conditioner PERSON every two days DATE . And I never, ever use a blowdryer without the diffuser attachment. / You're sweet. / I counted on you to help my cause. You and that thug are obviously failing. Aren't we ever going on our date? / Where? / How is our little Find the Wench A Date WORK_OF_ART plan progressing? / Forget French LANGUAGE . / I don't want to know how to say that though. I want to know useful things. Like where the good stores are. How much does champagne cost? Stuff like Chat. I have never in my life had to point out my head to someone. / C'esc ma tete. This is my head / Gosh, if only we could find Kat a boyfriend... / Unsolved mystery. She used to be really popular when she started high school, then it was just like she got sick of it or something. / The thing is, Cameron PERSON -- I'm at the mercy of a particularly hideous breed of loser. My sister. I can't date until she does. / No, no, it's my fault -- we didn't have a proper introduction --- / You're asking me out. That's so cute. What's your name again? / Not the hacking and gagging and spitting part. Please. / Can we make this quick? Roxanne Korrine PERSON and Andrew Barrett PERSON are having an incredibly horrendous public break- up on the quad. Again. / I did. / I have to be home in twenty minutes TIME . / Sometimes I wonder if the guys we're supposed to want to go out with are the ones we actually want to go out with, you know? / Combination. I don't know -- I thought he'd be different. More of a gentleman... / He practically proposed when he found out we had the same dermatologist. I mean. Dr. Bonchowski PERSON is great an all, but he's not exactly relevant party conversation. / Would you mind getting me a drink, Cameron PERSON ? / Joey PERSON . / Where did he go? He was just here. / You might wanna think about it / Did you change your hair? / You know the deal. I can ' t go if Kat PERSON doesn't go -- / Hi, Joey PERSON . / Neat... / Queen Harry? / Hopefully. / Expensive? / Patrick PERSON -- is that- a. / PERSON Is that woman a complete fruit-loop or is it just me? / No! I just wanted / I just wanted -- / Let go! / You looked beautiful last night TIME , you know. / I guess I'll never know, will I? / God, you're just like him! Just keep me locked away in the dark, so I can't experience anything for myself / I'm not stupid enough to repeat your mistakes. / No. you didn't! If you really thought I could make my own decisions, you would've let me go out with him instead of helping Daddy hold me hostage. / Why didn't you tell me? / But / You did what? / As in... / But you hate Joey PERSON / Why? / What? / I wish I had that luxury. I'm the only sophomore that got asked to the prom and I can't go, because you won ' t. / Like you care. / I don't get you. You act like you're too good for any of this, and then you go totally apeshit when you get here. / I really don't think I need any social advice from you right now. / You are so completely unbalanced. / Yeah, he's your freak friend Mandella PERSON 's boyfriend. I guess since I'm not allowed to go out, I should obsess over a dead guy, too. / Like I'm supposed to know what that even means. / Can't you forget for just one night TIME that you're completely wretched? / Bogey Lowenstein's ORG party is normal, but you're too busy listening to Bitches Who Need Prozac PERSON to know that. / You're ruining my life' Because you won't be normal, I can't be normal. / I think you're a freak. I think you do this to torture me. And I think you suck. / Oh, I thought you might have a date I don't know why I'm bothering to ask, but are you going to Bogey Lowenstein's ORG party Saturday night TIME ? / Oh my God, does this mean you're becoming normal? / Can you at least start wearing a bra? / Nowhere... Hi, Daddy. / I have a date, Daddy. And he ' s not a captain of oppression like some men we know. / Fine. I see that I'm a prisoner in my own house. I'm not a daughter. I'm a possession! / He's not a "hot rod". Whatever that is. / No, but / Daddy, I want to discuss the prom with you. It's tomorrow DATE night TIME -- / Why? / Daddy, no! / It's just a party. Daddy. / Daddy, people expect me to be there! / It's just a party. Daddy, but I knew you'd forbid me to go since " Gloria Steinem PERSON " over there isn't going -- / If you must know, we were attempting to go to a small study group of friends. / Daddy, I -- / But she doesn't want to date. / But it's not fair -- she's a mutant, Daddy! / What if she never starts dating? / Now don't get upset. Daddy, but there's this boy... and I think he might ask...

Exploring Different Types of Entities

# Count and sort the number of characters with lines of each movie
movie.groupby('mname')['mid'].count().sort_values(ascending=False)

mname
magnolia                          18
lone star                         16
the anniversary party             12
nixon                             12
grand hotel                       11
                                  ..
dark city                          1
quantum project                    1
the nightmare before christmas     1
metropolis                         1
predator                           1
Name: mid, Length: 600, dtype: int64

# Count the number of entities in the lines of the characters in the movie magnolia
ent_types = dict() # initialize a dictionary
for line in movie[movie['mname']=='magnolia']['lines']: # loop through the lines in "magnolia"
    doc = nlp(line)
    for entity in doc.ents: # for each character, loop through all the entities
        label = entity.label_ # get their labels
        if label not in ent_types: # make sure there's a key for this label in the dictionary
            ent_types[label] = Counter() # each label key points to a Counter for examples
        text = entity.text
        ent_types[label][text] += 1 # count the number of times we see each example

# Count of each type of entities
for etype, examples in ent_types.items():
    print(etype, len(examples))

PERSON 73
GPE 12
ORG 31
TIME 11
CARDINAL 19
DATE 27
NORP 2
WORK_OF_ART 8
ORDINAL 1
QUANTITY 3
LAW 1
FAC 2

# Explain the entity type "PERSON"
spacy.explain('PERSON')

'People, including fictional'

# Get all people
people = []
for line in movie[movie['mname']=='magnolia']['lines']:
    doc = nlp(line)
    for named_entity in doc.ents:
        if named_entity.label_ == "PERSON": # we only want the labels with a type "PERSON"
            people.append(named_entity.text)
people_count = Counter(people)
# sort the people names by their occurences, and convert the results into a dataframe
people_magnolia = pd.DataFrame(people_count.most_common(), columns=['character', 'count'])
people_magnolia

	character	count
0	Linda	15
1	Frank	14
2	Phil	7
3	Earl	6
4	Jimmy	6
...	...	...
68	I'M SMART	1
69	Jean Baptiste	1
70	Willa Cather	1
71	Dad	1
72	Picky	1

73 rows × 2 columns

# Explain a certain type of entity "TIME" and "DATE"
print("TIME:", spacy.explain('TIME'))
print("DATE:", spacy.explain('DATE'))

TIME: Times smaller than a day
DATE: Absolute or relative dates or periods

Task 1

# Get entities related to time and their numbers in the movie magnolia and convert the sorted results to a dataframe.
# Like what we did for "people", first create an empty list, and then loop through each line in the movie
# and use "nlp" to convert the line. Next, loop through each entity of the line, and select the entity based on labels
# "TIME" or "DATE", and append the result to the empity list. Finally, use Counter to count the results,
# create a dataframe and sort them using "most_common."

time = []
for line in movie[movie['mname']=='magnolia']['lines']:
    doc = nlp(line)
    for named_entity in doc.ents:
        if named_entity.label_ == "TIME" or named_entity.label_ == "DATE":
            time.append(named_entity.text)
time_count = Counter(time)
time_magnolia = pd.DataFrame(time_count.most_common(), columns=['character', 'count'])
print(time_magnolia)

                  character  count
0                     today      6
1                   tonight      4
2                last night      3
3                four years      2
4               ten o'clock      2
5                   an hour      2
6                  a minute      1
7             Eight o'clock      1
8       About two years ago      1
9                  tomorrow      1
10             seven-eleven      1
11               age twelve      1
12            age seventeen      1
13       thirty eight years      1
14             12 years old      1
15                      '84      1
16               one minute      1
17               five years      1
18                     1980      1
19          three years ago      1
20              Ten o'clock      1
21                    night      1
22           two and a half      1
23                    a day      1
24        about three years      1
25             thirty years      1
26                  ' years      1
27         twenty years old      1
28      Fifteen minutes ago      1
29            ten years ago      1
30             this morning      1
31  Two years...three years      1
32          fifty years old      1
33                 two days      1
34    About three weeks ago      1
35               six months      1
36                   ' days      1
37          ten minutes ago      1

# Get the total word count of all lines of movies released in the year of 1960
movie_1960_df = movie[(movie['year']==1960)]
movie_1960_wordcount = movie_1960_df["wordcount"].sum()
movie_1960_wordcount

19535

# Get the total word count of all lines of movies released in the year of 2009
movie_2009_df = movie[(movie['year']==2009)]
movie_2009_wordcount = movie_2009_df["wordcount"].sum()
movie_2009_wordcount

15858

Task 2

# Calculate the number of entities related to time in movies released in 1960 and 2009, and divide by respective word counts.
# First, initialize a variable "time_1960_count" and set it as 0. Then like what we did before, loop through each line in 
# movie_1960_df['lines'], and use "nlp" to convert the line. Next, loop through each entity of the line, 
# and if the entity has a label of "TIME" or "DATE", add 1 to time_1960_count. Finally, print time_1960_count,
# divided by movie_1960_wordcount. And then repeat the process for 2009 movies.

time_1960_count = 0
for line in movie[movie['year']==1960]['lines']:
    doc = nlp(line)
    for named_entity in doc.ents:
        if named_entity.label_ == "TIME" or named_entity.label_ == "DATE":
            time_1960_count += 1 
print('1960:',time_1960_count/movie_1960_wordcount)

time_2009_count = 0
for line in movie[movie['year']==2009]['lines']:
    doc = nlp(line)
    for named_entity in doc.ents:
        if named_entity.label_ == "TIME" or named_entity.label_ == "DATE":
            time_2009_count += 1 
print('2009:', time_2009_count/movie_2009_wordcount)

1960: 0.008395188123880215
2009: 0.004540295119182747

Part-of-Speech (POS) Tagging

Parts of speech are the grammatical units of language, such as (in English) nouns, verbs, adjectives, adverbs, pronouns, and prepositions. Each of these parts of speech plays a different role in a sentence. By computationally identifying parts of speech, we can start computationally exploring syntax, the relationship between words, rather than only focusing on words in isolation, as we did with tf-idf.

# Get the POS tagging of a sample text
sample = """Or set upon a golden bough to sing to lords and ladies of Byzantium of what is past, or passing, or to come."""
# This is an excerpt from "Sailing to Byzantium" by the Irish poet W. B. Yeats
document = nlp(sample)
options = {"compact": True, "distance": 90, "color": "yellow", "bg": "black", "font": "Gill Sans"}
displacy.render(document, style="dep", options=options) # visualize it

# Get part-of-speech tags
for token in document:
    print(token.text, token.pos_, token.dep_) # pos_ means part-of-speech tags, and dep_ means dependency

Or CCONJ cc
set VERB ROOT
upon SCONJ prep
a DET det
golden ADJ amod
bough NOUN pobj
to PART aux
sing VERB advcl
to ADP prep
lords NOUN pobj
and CCONJ cc
ladies NOUN conj
of ADP prep
Byzantium PROPN pobj
of ADP prep
what PRON nsubj
is AUX pcomp
past ADJ acomp
, PUNCT punct
or CCONJ cc
passing VERB conj
, PUNCT punct
or CCONJ cc
to PART aux
come VERB conj
. PUNCT punct

# Get verbs from the movie "magnolia" in the movie dialogue dataset

verbs = []
for line in movie[movie['mname']=='magnolia']['lines']:
    doc = nlp(line)
    for token in doc: # loop through the token, instead of entities
        if token.pos_ == 'VERB': # we only want the tokens with a POS tagging of "VERB"
            verbs.append(token.text)
verbs_count = Counter(verbs)
# sort the verb by their occurences, and convert the results into a dataframe
verbs_magnolia = pd.DataFrame(verbs_count.most_common(), columns=['verb', 'count'])
verbs_magnolia

	verb	count
0	know	80
1	have	79
2	do	75
3	go	50
4	want	49
...	...	...
460	drink	1
461	presume	1
462	spoken	1
463	Sounds	1
464	threatening	1

465 rows × 2 columns

Task 3

# Get the top 10 adjectives by count in movies released in 1960 and 2009.
# Like what we did for the movie "magnolia", first create an empty list of adjectives, and then loop through each line in 
# movie_1960_df['lines'], and use "nlp" to convert the line. Next, loop through each token of the line, and select the entity 
# based on POS tagging "ADJ", and append the result to the empity list. Finally, use Counter to count the results,
# create a dataframe and sort them using "most_common," and print the top 10 adjectives.
# And then repeat the process for 2009 movies.

adjectives = []
for line in movie[movie['year']==1960]['lines']:
    doc = nlp(line)
    for token in doc:
        if token.pos_ == 'ADJ':
            adjectives.append(token.text)
adj_count = Counter(adjectives)
adj_1960 = pd.DataFrame(adj_count.most_common(), columns=['adjectives', 'count'])
print('1960', adj_1960[:10])

adjectives2 = []
for line in movie[movie['year']==2009]['lines']:
    doc = nlp(line)
    for token in doc:
        if token.pos_ == 'ADJ':
            adjectives2.append(token.text)
adj2_count = Counter(adjectives2)
adj_2009 = pd.DataFrame(adj2_count.most_common(), columns=['adjectives', 'count'])
print('2009', adj_2009[:10])

1960   adjectives  count
0     little     35
1      right     26
2      sorry     23
3      wrong     23
4       good     20
5      other     19
6        old     18
7       Good     17
8       sure     17
9       wise     16
2009   adjectives  count
0       more     22
1       dead     19
2       good     19
3     little     17
4      other     15
5      right     15
6      ready     12
7       Good     11
8        bad     11
9       next     10

Keyword Extraction

import re # regular expression
from IPython.display import Markdown, display # for visualization

# Visualize the data from the movie Casablanca
movie_casablanca = movie[movie['mname']=='casablanca'].reset_index(drop=True)
for line in movie_casablanca['lines']:
    displacy.render(nlp(line), style="ent")

Monsieur Rick PERSON , I -- / Oh, but if you knew what it means to us to leave Europe LOC , to get to America GPE ! Oh, but if Jan PERSON should find out! He is such a boy. In many ways I am so much older than he is. / Oh, yes, please. / And he never knew, and the girl kept this bad thing locked in her heart? That would be all right, wouldn't it? / -- Will he keep his word? / Yes, Monsieur. / Oh, yes. / He tells me he can give us an exit visa, but we have no money. / Yes, but we have not much money, and traveling is so expensive and difficult. It was much more than we thought to get here. And then Captain Renault PERSON sees us and he is so kind. He wants to help us. / Eight weeks DATE . We come from Bulgaria GPE . Oh, things are very bad there, Monsieur. A devil has the people by the throat. So, Jan PERSON and I, we, we do not want our children to grow up in such a country. / At the roulette table, trying to win enough for our exit visa. Well of course, he's losing. / He did. Captain Renault PERSON did. / No, I mean, is he trustworthy? Is his word -- / Monsieur Rick PERSON , what kind of man is Captain Renault PERSON ? / No. / My husband is with me, too. / I came with Captain Renault PERSON . / Could I speak to you for just a moment, please? / Monsieur Rick PERSON ?

I shall remember to pay it... to myself. / Of course they stay. Rick PERSON 's wouldn't be Rick PERSON 's without them. / Hmmm. I happen to know that he gets ten percent PERCENT . But he's worth twenty- five CARDINAL . / Ah, to get out of Casablanca GPE and go to America GPE ! You're a lucky man. / Shall we draw up the papers, or is our handshake good enough? / Rick PERSON , don't be a fool. Take me into your confidence. You need a partner. / Rick PERSON , I'll put my cards on the table. I think you know where those letters are. / Naturally there will be a few incidental expenses. That is the proposition I have for whoever has those letters. / I have a proposition for whoever has those letters. I will handle the entire transaction, get rid of the letters, take all the risk, for a small percentage. / If I could lay my hands on those letters, I could make a fortune. / Of course not. What upsets me is the fact that Ugarte ORG is dead and no one knows where those letters of transit are. / The bourbon. The news about Ugarte ORG upset me very much. / Carrying charges, my boy, carrying charges. Here, sit down. There's something I want to talk over with you, anyhow. / No hurry. I'll have it sent over. Have a drink with me. / My dear Rick PERSON , when will you realize that in this world today DATE isolationism is no longer a practical policy? / Suppose we ask Sam PERSON . Maybe he'd like to make a change. / That's too bad. That's Casablanca PERSON 's leading commodity. In refugees alone we could make a fortune if you would work with me through the black market. / What do you want for Sam PERSON ? / You haven't heard my offer. / Fine, but I would like to buy your cafe. / Hello, Rick PERSON . / It was gracious of you to share it with me. Good day DATE , Mademoiselle ORG , Monsieur. / He is a difficult customer, that Rick PERSON . One CARDINAL never knows what he'll do or why. But it is worth a chance. / Not for sure, Monsieur, but I will venture to guess that Ugarte ORG left those letters with Monsieur Rick PERSON . / Those letters were not found on Ugarte ORG when they arrested him. / I observe that you in one respect are a very fortunate man, Monsieur PERSON . I am moved to make one CARDINAL more suggestion, why, I do not know, because it cannot possibly profit me, but, have you heard about Signor Ugarte PERSON and the letters of transit? / Well, good luck. But be careful. You know you're being shadowed? / -- We might as well be frank, Monsieur. It will take a miracle to get you out of Casablanca GPE . And the Germans NORP have outlawed miracles. / As leader of all illegal activities in Casablanca GPE , I am an influential and respected man. It would not be worth my life to do anything for Monsieur Laszlo. You, however, are a different matter.

And the other alternative? / What are they? / What value is that? You may recall what German NORP guarantees have been worth in the past. / Occupied France GPE ? / This morning TIME you implied it was not safe for him to leave Casablanca WORK_OF_ART . / You needn't be concerned about me. / Yes. / And I said I would never leave you. / But what about us? / No. / But Richard PERSON , no, I, I -- / No, Richard PERSON , no. What has happened to you? Last night TIME we said -- / I don't understand. What about you? / But why my name, Richard PERSON ? / Oh, Rick PERSON ! / But it's all right, isn't it? You were able to arrange everything? / Richard PERSON , Victor PERSON thinks I'm leaving with him. Haven't you told him? / I wish I didn't love you so much. / I can't fight it anymore. I ran away from you once. I can't do it again. Oh, I don't know what's right any longer. You'll have to think for both of us, for all of us. / Oh, you'll help him now, Richard PERSON , won't you? You'll see that he gets out? Then he'll have his work, all that he's been living for. / Now? I don't know. I know that I'll never have the strength to leave you again. / Just before you and I were to leave Paris GPE together. A friend came and told me that Victor PERSON was alive. They were hiding him in a freight car on the outskirts of Paris GPE . He was sick, he needed me. I wanted to tell you, but I, I didn't care. I knew, I knew you wouldn't have left Paris GPE , and the Gestapo PERSON would have caught you. So I... well, well, you know the rest. / Oh, it wasn't my secret, Richard PERSON . Victor PERSON wanted it that way. Not even our closest friends knew about our marriage. That was his way of protecting me. I knew so much about his work, and if the Gestapo PERSON found out I was his wife it would be dangerous for me and for those working with me. / It wasn't long after we were married that Victor PERSON went back to Czechoslovakia GPE . They needed him in Prague GPE , but there the Gestapo PERSON were waiting for him. Just a two CARDINAL -line item in the paper: " Victor Laszlo PERSON apprehended. Sent to concentration camp." I was frantic. For months DATE I tried to get word. Then it came. He was dead, shot trying to escape. I was lonely. I had nothing. Not even hope. Then I met you. / For the last time, put them on the table. / Put them on the table. / -- All right. I tried to reason with you. I tried everything. Now I want those letters. Get them for me. / No. Oh, Richard PERSON , I'm sorry. I'm sorry, but, but you, you are our last hope. If you don't help us, Victor Laszlo PERSON will die in Casablanca GPE . / Please. Please listen to me. If you knew what really happened, if you only knew the truth -- / Richard PERSON , Richard PERSON , we loved each other once. If those days DATE meant anything at all to you -- / It was your cause, too. In your own way, you were fighting for the same thing. / I know how you feel about me, but I'm asking you to put your feelings aside for something more important. / You can ask any price you want, but you must give me those letters. / Please. / Richard PERSON , I had to see you. / The stairs from the street. / No, Rick PERSON . No, you see, Victor Laszlo PERSON is my husband... and was, even when I knew you in Paris GPE . / You can believe that if you want to. / Last night TIME I saw what has happened to you. The Rick PERSON I knew in Paris GPE , I could tell him. He'd understand. But the one who looked at me with such hatred... well, I'll be leaving Casablanca WORK_OF_ART soon and we'll never see each other again. We knew very little about each other when we were in love in Paris GPE . If we leave it that way, maybe we'll remember those days and not Casablanca PERSON , not last night TIME . / I don't think I will, Rick PERSON . / Yes. / It doesn't matter. / It's about a girl who had just come to Paris GPE from her home in Oslo GPE . At the house of some friends she met a man about whom she'd heard her whole life, a very great and courageous man. He opened up for her a whole beautiful world full of knowledge and thoughts and ideals. Everything she knew or ever became was because of him. And she looked up to him and worshipped him with a feeling she supposed was love. / I don't know the finish yet. / Can I tell you a story, Rick PERSON ? / I didn't count the days DATE . / Please don't. Don't, Rick PERSON . I can understand how you feel. / I wouldn't have come if I had known that you were here. Believe me, Rick PERSON , it's true. I didn't know. / Please. / No. No, Rick PERSON . Not tonight TIME . / I love you so much, and I hate this war so much. Oh, it's a crazy world. Anything can happen. If you shouldn't get away, I mean, if, if something should keep us apart, wherever they put you and wherever I'll be, I want you to know... / Oh, darling! / That's too far ahead to plan. / No, no. Not at my hotel. I, uh, I have things to do in the city before I leave. I'll meet you at the station, huh? / Yes, of course, we -- / But be serious, darling. You are in danger and you must leave Paris GPE . / Strange. I know so very little about you. / Was that cannon fire, or is it my heart pounding? / Ten years ago DATE ? Let's see... ...Yes. I was having a brace put on my teeth. Where were you? / With the whole world crumbling, we pick this time to fall in love. / It's the Gestapo PERSON . They say they expect to be in Paris GPE tomorrow DATE . / Richard PERSON , they'll find out your record. It won't be safe for you here. / Well, only one CARDINAL answer can take care of all our questions. / That's easy. There was. He's dead. / Why there is no other man in my life? / Yes? / I'm willing to be overcharged. Tell me. / A franc for your thoughts. / We said "no questions." / There's still nobody in the world who can play "As Time Goes By WORK_OF_ART " like Sam PERSON . / Say goodnight to Sam PERSON for me. / Yes. I put that dress away. When the Germans NORP march out, I'll wear it again. / No. / How nice. You remembered. But of course, that was the day DATE the Germans NORP marched into Paris GPE . / Play it, Sam PERSON . Play " As Time Goes By WORK_OF_ART ." / Play it once, Sam PERSON , for old time's sake. / You used to be a much better liar, Sam PERSON . / Does he always leave so early? / When will he be back? / Where is Rick PERSON ? / Some of the old songs, Sam PERSON . / It's been a long time. / Hello, Sam PERSON . / Yes, I'm ready. Goodbye, Rick PERSON . God PERSON bless you. / Be careful. / Yes, Yes I know. Victor PERSON , whatever I do, will you believe that I, that -- / No, Victor PERSON , there isn't. / Yes, Victor PERSON , I was. / -- Yes? / Ask me? / Did he give any reason? / Yes? / Whatever I'd say, you'd carry on. Victor PERSON , why don't you tell me about Rick PERSON ? What did you find out? / Don't joke. After Major Strasser PERSON 's warning tonight TIME , I am frightened. / Victor, please, don't go to the underground meeting tonight TIME . / What happened with Rick PERSON ? / He does. Could we have a table close to him? / Yes, I see. When I had trouble getting out of Lille GPE , why didn't you leave me there? And when I was sick in Marseilles GPE and held you up for two weeks DATE and you were in danger every minute TIME of the time, why didn't you leave me then? / But, Victor PERSON , if the situation were different, if I had to stay and there were only a visa for one, would you take it? / No, Victor PERSON , no. / We are only interested in two CARDINAL visas, Signor PERSON . / Goodnight. / Goodnight. / Be careful. / Victor, I'm afraid for you. / You are very kind. / Victor, I, I feel somehow we shouldn't stay here. / You're saying this only to make me go. / I wasn't sure you were the same. Let's see, the last time we met -- / -- This is Mr. Laszlo PERSON . / Hello, Rick PERSON . / -- Is what? / Rick PERSON ? Who's he? / Yes. / Captain, the boy who is playing the piano, somewhere I have seen him.

I see. / As you see, it was true every single time. Thank heaven I found you, Berger PERSON . I am looking for a man by the name of Ugarte ORG . He is supposed to help me. / In a concentration camp, one is apt to lose a little weight. / A champagne cocktail, please. / Mr. Berger PERSON , the ring, could I see it again? / I'm sorry. It is. / What is your name? / Thank you, but I hardly think -- / Well? / Good day. / Thank you very much. Good day DATE . / Rick PERSON ? / Do you know where they are? / Yes, something. / We've decided, Signor Ferrari PERSON . For the present we'll go on looking for two CARDINAL exit visas. Thank you very much. / I will stay here and keep on trying. I'm sure in a little while -- / Signor Ferrari PERSON thinks it might just be possible to get an exit visa for you. / You see, my dear, the word has gone around. / Are you ready Ilsa? / Of course, I'll be careful. / -- You don't even have to say it. I'll believe. Goodnight, dear. / I love you very much, my dear. / I know how it is to be lonely. Is there anything you wish to tell me? / When I was in the concentration camp, were you lonely in Paris GPE ? / Ilsa, I -- / Yes. He said, "Ask your wife." I don't know why he said that. / He suggested I ask you. / But no intention of selling them. One would think if sentiment wouldn't persuade him, money would. / Apparently he has the letters. / To tell you the truth, I am frightened too. Shall I remain here in our hotel room hiding, or shall I carry on the best I can? / I must. Besides, it isn't often that a man has a chance to display heroics before his wife. / Our faithful friend is still there. / We'll discuss it later. / And as far away from Major Strasser PERSON as possible. / I meant to, but something always held me up. I love you very much, Ilsa GPE . / Yes, I would. / No, Ilsa GPE , I won't let you stay here. You must get to America GPE . And believe me, somehow I will get out and join you. / Please, Ilsa GPE , don't be hasty. / Goodnight. / Goodnight. / I will, don't worry. / I must find out what Berger PERSON knows. / We have been in difficult places before, haven't we? / This time they really mean to stop me. / Won't you join us? / If we would walk out so soon, it would only call attention to us. Perhaps Ugarte ORG 's in some other part of the cafe. / I understand. / Yes. / Monsieur Blaine PERSON , I don't ask you to explain anything. / Everything in order? / No. It's all arranged. / But we made a deal. / I brought the money, Monsieur Blaine PERSON . / Monsieur Blaine PERSON , I don't know how to thank you. / Apparently you think of me only as the leader of a cause. Well, I am also a human being. / I know a good deal more about you than you suspect. I know, for instance, that you are in love with a woman. It is perhaps strange that we both should be in love with the same woman. The first evening TIME I came here in this cafe, I knew there was something between you and Ilsa GPE . Since no one is to blame, I, I demand no explanation. I ask only one CARDINAL thing. You won't give me the letters of transit. All right. But I want my wife to be safe. I ask you as a favor to use the letters to take her away from Casablanca GPE . / I wonder if you do. I wonder if you know that you're trying to escape from yourself and that you'll never succeed. / You know how you sound, Monsieur Blaine PERSON ? Like a man who's trying to convince himself of something he doesn't believe in his heart. Each of us has a destiny, for good or for evil. / We might as well question why we breathe. If we stop breathing, we'll die. If we stop fighting our enemies, the world will die. / Yes, rather. / Thank you. / My wife? / I beg your pardon? / There must be some reason why you won't let me have them. / I'll raise it to two hundred thousand CARDINAL . / Are you enough of a businessman to appreciate an offer of a hundred thousand francs MONEY ? / Isn't it strange that you always happened to be fighting on the side of the underdog? / My friends in the underground tell me that you have quite a record. You ran guns to Ethiopia GPE . You fought against the fascists in Spain GPE . / You must know it's very important I get out of Casablanca GPE . It's my privilege to be one of the leaders of a great movement. You know what I have been doing. You know what it means to the work, to the lives of thousands and thousands CARDINAL of people that I be free to reach America GPE and continue my work. / Right. / Well, isn't there some other place? It's rather confidential, what I have to say. / Monsieur Blaine PERSON , I wonder if I could talk to you? / Good evening TIME . You see, here we are again. / Good morning. / We'll come again. / I hope we didn't overstay our welcome. / Thank you. I try. / What for? / Won't you join us for a drink? / One hears a great deal about Rick PERSON in Casablanca GPE . / How do you do? / Are you quite finished with us? / Nothing important, but may I speak to him now? / Yes. / -- Thank you. / I was in a German NORP concentration camp for a year DATE . That's honor enough for a lifetime. / Well, perhaps I shall like it in Casablanca GPE . / Thank you. / May we proceed with the business? / I slept very well. / We'll be there. / Ilsa, I don't wish to be the one to say it, but it's late. / This is a very interesting cafe. I congratulate you. / My bill. / Very well. / No, Captain, please. / Thank you. I hope you'll forgive me, Captain PERSON , but the present French NORP administration has not always been so cordial. May I present Miss Ilsa Lund? / Yes. What is it you want? / Yes. / Good day. / You won't dare to interfere with me here. This is still unoccupied France GPE . Any violation of neutrality would reflect on Captain Renault PERSON . / And what if you track down these men and kill them? What if you murdered all of us? From every corner of Europe LOC , hundreds CARDINAL , thousands CARDINAL , would rise to take our places. Even Nazis NORP can't kill that fast. / If I didn't give them to you in a concentration camp where you had more "persuasive methods" at your disposal, I certainly won't give them to you now. / Even in Berlin GPE . / And that is? / Is that all you wish to tell us? / Whether or not you succeed is, of course, problematical. / Captain Renault PERSON , I am under your authority. Is it your order that we come to your office? / This is hardly the time or the place. / I've never accepted that privilege, and I'm now on French NORP soil. / I'm sure you'll excuse me if I am not gracious, but you see, Major Strasser PERSON , I'm a Czechoslovakian NORP .

I'm afraid Major Strasser PERSON would insist. / I can't get over you two CARDINAL . She was asking about you earlier, Rick PERSON , in a way that made me extremely jealous. / Well then, perhaps you also --- / Oh, you've already met Rick PERSON , Mademoiselle PERSON ? / Well, Mademoiselle ORG , he's the kind of a man that, well, if I were a woman and I... were not around, I should be in love with Rick PERSON . But what a fool I am talking to a beautiful woman about another man. / Mademoiselle, you are in Rick PERSON 's and Rick PERSON is -- / He came from Paris GPE with Rick PERSON . / Sam PERSON ? / I am making out the report now. We haven't quite decided whether he committed suicide or died trying to escape. / I believe you have a message for him? / By the way, Monsieur PERSON , last night TIME you evinced an interest in Signor Ugarte GPE . / Monsieur, insofar as it is in my power -- / And the honor of having served the Third Reich FAC . / I am afraid not. My regrets, Monsieur PERSON . / With pleasure. Won't you sit down? / That's strange. Nobody is supposed to sleep well in Casablanca GPE . / I am delighted to see you both. Did you have a good night's rest? / Goodnight. / Tomorrow DATE at ten at the Prefect FAC 's office. / So it is. And we have a curfew here in Casablanca GPE . It would never do for the Chief of Police to be found drinking after hours TIME and have to fine himself. / Ricky, you're becoming quite human. I suppose we have to thank you for that, Mademoiselle ORG . / Well! A precedent is being broken. Er, Emil! / No. Two CARDINAL champagne cocktails, please. / Let us say that it is my request. That is a much more pleasant word. / No. Please, Monsieur, it is a little game we play. They put it on the bill, I tear the bill up. It is very convenient. / I was informed you were the most beautiful woman ever to visit Casablanca GPE . That was a gross understatement. / Merely to welcome you to Casablanca PERSON and wish you a pleasant stay. It is not often we have so distinguished a visitor. / I am Captain Renault PERSON , Prefect of Police WORK_OF_ART . / Monsieur Laszlo PERSON , is it not? / Ask Monsieur Rick PERSON . / Victor Laszlo PERSON is on that plane. / But I have no excuse to close it. / But everybody's having such a good time. / I see what you mean. / A leader, like Laszlo PERSON ? / I have no conviction, if that's what you mean. I blow with the wind, and the prevailing wind happens to be from Vichy. / My dear Major, we are trying to cooperate with your government, but we cannot regulate the feelings of our people. / It may interest you to know that at this very moment he is on his way here. / But we mustn't underestimate American NORP blundering. I was with them when they "blundered" into Berlin GPE in 1918 DATE . / If Rick PERSON has the letters, he's much too smart to let you find them there. / Mademoiselle. / You see, Major, you have nothing to worry about Rick PERSON . / Of course, one must admit he has great courage. / Rick PERSON is completely neutral about everything. And that takes in the field of women, too. / Ho, diplomatist! / Well, personally, Major PERSON , I will take what comes. / Rick PERSON , this is Major Heinrich Strasser PERSON of the Third Reich ORG . / Especially so tonight TIME , Major PERSON . In a few minutes TIME you will see the arrest of the man who murdered your couriers. / May I recommend Veuve Cliquot ' PERSON 26 CARDINAL , a good French NORP wine. / Thank you. It is a pleasure to have you here, Major PERSON . / Good evening TIME , gentlemen. / Oh, there is no hurry. Tonight TIME he'll be at Rick PERSON 's. Everybody comes to Rick PERSON 's. / Realizing the importance of the case, my men are rounding up twice the usual number of suspects. / What else, my dear Major? / You may find the climate of Casablanca GPE a trifle warm, Major PERSON . / Major Strasser PERSON , my aide, Lieutenant Casselle WORK_OF_ART . / Unoccupied France GPE welcomes you to Casablanca GPE . / Uh huh. / And that ten thousand francs MONEY should pay our expenses. / It might be a good idea for you to disappear from Casablanca GPE for a while. There's a Free French NORP garrison over at Brazzaville GPE . I could be induced to arrange a passage. / I think perhaps you're right. / Well, Rick PERSON , you're not only a sentimentalist, but you've become a patriot. / I suppose you know this isn't going to be pleasant for either of us, especially for you. I'll have to arrest you of course. / What you just did for Laszlo PERSON , and that fairy tale that you invented to send Ilsa GPE away with him. I know a little about women, my friend. She went, but she knew you were lying. / Well I was right. You are a sentimentalist. / You think of everything, don't you? / That is my least vulnerable spot. / Call off your watch-dogs you said. / I suppose you know what you're doing, but I wonder if you realize what this means? / Put that gun down. / Have you taken leave of your senses? / Serves me right for not being musical. / Tell me, when we searched the place, where were they? / Is everything ready? / You know, this place will never be the same without you, Ricky PERSON . / Oh, he won't be followed here. / I was informed just as Laszlo PERSON was about to leave the hotel, so I knew I'd be on time. / Go ahead, Ricky PERSON . / Ricky, I'm going to miss you. Apparently you're the only one in Casablanca GPE who has even less scruples than I. / I see. How do I know you'll keep your end of the bargain? / There's still something about this business I don't quite understand. Miss Lund PERSON , she's very beautiful, yes, but you were never interested in any woman. / It certainly would. Germany GPE ... Vichy would be very grateful. / You didn't come here to tell me this. You have the letters of transit. You can fill in your name and hers and leave any time you please. Why are you interested in what happens to Laszlo PERSON ? / What friend? / Huh? / Don't count too much on my friendship, Ricky PERSON . In this matter I'm powerless. Besides, I might lose ten thousand francs MONEY . / Because one, you've bet ten thousand francs MONEY he'd escape. Two CARDINAL , you have the letters of transit, now don't bother to deny it. And, well, you might do it simply because you don't like Strasser PERSON 's looks. As a matter of fact, I don't like him either. / Ricky, I'd advise you not to be too interested in what happens to Laszlo PERSON . If by any chance you were to help him escape -- / I am shocked, shocked to find that gambling is going on in here! / Well, I forgive you this time. But I'll be in tomorrow DATE night with a breathtaking blonde, and it will make me very happy if she loses. Uh huh! / Why do you interfere with my little romances? / As I suspected, you're a rank sentimentalist. / Who knows? In her own way she may constitute an entire second ORDINAL front. I think it's time for me to flatter Major Strasser PERSON a little. I'll see you later, Rick PERSON . / Huh? / Serves me right for asking a direct question. The subject is closed. / Well, I told Strasser PERSON he wouldn't find the letters here. But I told my men to be especially destructive. You know how that impresses Germans NORP ? Rick PERSON , have you got these letters of transit? / Well, Ricky PERSON . I'm very pleased with you. Now you're beginning to live like a Frenchman NORP . / Another precedent gone. This has been a very interesting evening TIME . I'll call you a cab. Gasoline rationing, time of night. / Oh, no, Rick PERSON never -- / Well, you were asking about Rick PERSON and here he is. Mademoiselle, may I present -- / That makes Rick PERSON a citizen of the world. / And you already know Herr Heinze PERSON of the Third Reich ORG . / Excuse me. / I have my orders. / The winning side would have paid you much better. / Oh, laugh if you will, but I happen to be familiar with your record. Let me point out just two CARDINAL items. In 1935 DATE you ran guns to Ethiopia GPE . In 1936 DATE , you fought in Spain GPE on the Loyalist ORG side. / Because, my dear Ricky PERSON , I suspect that under that cynical shell you're at heart a sentimentalist. / It doesn't matter. There is no exit visa for him. / I think not. I have seen the lady. And if he did not leave her in Marseilles GPE , or in Oran GPE , he certainly won't leave her in Casablanca GPE . / He is traveling with a lady. / Done. No matter how clever he is, he still needs an exit visa, or I should say, two CARDINAL . / Make it ten. I am only a poor corrupt official. / Is that a serious offer? / This is the end of the chase. / Oh, but I just told you. -- / Manages what? / It is my duty to see that he doesn't impress the other half CARDINAL . Rick PERSON , Laszlo PERSON must never reach America GPE . He stays in Casablanca GPE . / Rick PERSON , that is the first ORDINAL time I have ever seen you so impressed. / Victor Laszlo PERSON . / That is another reason. There is a man who's arrived in Casablanca GPE on his way to America GPE . He will offer a fortune to anyone who will furnish him with an exit visa. / Thank you. Rick PERSON , there are many exit visas sold in this cafe, but we know that you have never sold one. That is the reason we permit you to remain open. / How observant you are. As a matter of fact, I wanted to give you a word of advice. / Perhaps not. / You know, Rick PERSON , we could have made this arrest earlier in the evening TIME at the Blue Parrot FAC , but out of my high regard for you we are staging it here. It will amuse your customers. / A wise foreign policy. / If you are thinking of warning him, don't put yourself out. He cannot possibly escape. / This is no ordinary arrest. A murderer, no less. / Rick PERSON , there's going to be some excitement here tonight TIME . We are going to make an arrest in your cafe. / Huh! / Waters? What waters? We're in the desert. / And what in heaven's name brought you to Casablanca PERSON ? / The clipper to America GPE . / The plane to Lisbon GPE . You would like to be on it? / You know, I think now I shall pay a call on Yvonne GPE , maybe get her on the rebound, eh? / How extravagant you are, throwing away women like that. Someday they may be scarce.

-- He's just a lucky guy. / Yes, well, everybody in Casablanca GPE has problems. Yours may work out. You'll excuse me. / Go back to Bulgaria GPE . / You want my advice? / Nobody ever loved me that much. / He always has. / And you want to know -- / And he is still willing to give you a visa? / Does he know that? / Yes, I'll bet. / So you decided to go to America GPE . / How long have you been married? / I thought so. Where's your husband? / -- Now, just a minute TIME . Who told you to ask me that? / Oh, he's just like any other man, only more so. / No, of course not. Do you mind if I do? / He is? Well, Captain Renault PERSON 's getting broadminded. Sit down. Will you have a drink? / I should have known. / How did you get in here? You're under age. / Yes? / I want you to take Miss Lund PERSON home. / -- Tonight TIME he forgot. / I want you to turn out the light in the rear entrance. It might attract the police. / Come up here a minute TIME . / Goodnight. / -- Don't tell me where you're going. / Now you finish locking up, will you, Carl PERSON ? / Maybe I won't have to. A bribe has worked before. In the meantime, everybody stays on salary. / How long can I afford to stay closed? / No thanks, Carl PERSON . / That's all right. / Here you are. / -- Forget it, Emil ORG . Mistakes like that happen all the time. / Well, I'll get it from the safe. / Don't forget, you owe Rick PERSON 's a hundred CARDINAL cartons of American NORP cigarettes. / Well, so long. / And Abdul PERSON and Carl PERSON and Sacha, they stay with the place, or I don't sell. / Oh, by the way, my agreement with Sam PERSON 's always been that he gets twenty- five percent PERCENT of the profits. That still goes. / It's certainly not good enough. But since I'm in a hurry, it'll have to do. / That's why I came over here to give them a chance to ransack my place. / Well, you're in good company. Renault ORG and Strasser PERSON probably think so, too. / I'll tell him when he comes in. / And the carrying charges? / So could I. And I'm a poor businessman. / Practically no one. / You're a fat hypocrite. You don't feel any sorrier for Ugarte ORG than I do. / I never drink in the morning. And every time you send my shipment over, it's always just a little bit short. / I see the bus is in. I'll take my shipment with me. / Suppose we do. / Suppose you run your business and let me run mine. / I don't buy or sell human beings. / It's not for sale at any price. / It's not for sale. / Hello, Ferrari PERSON . How's business at the Blue Parrot FAC ? / She tried everything to get them, and nothing worked. She did her best to convince me that she was still in love with me, but that was all over long ago. For your sake, she pretended it wasn't, and I let her pretend. / And you never will. But I've got a job to do, too. Where I'm going you can't follow. What I've got to do you can't be any part of. Ilsa, I'm no good at being noble, but it doesn't take much to see that the problems of three CARDINAL little people don't amount to a hill of beans in this crazy world. Someday you'll understand that. Now, now... / We'll always have Paris GPE . We didn't have, we'd lost it, until you came to Casablanca GPE . We got it back last night TIME . / Maybe not today DATE , maybe not tomorrow DATE , but soon, and for the rest of your life. / I'm saying it because it's true. Inside of us we both know you belong with Victor PERSON . You're part of his work, the thing that keeps him going. If that plane leaves the ground and you're not with him, you'll regret it. / -- You've got to listen to me. Do you have any idea what you'd have to look forward to if you stayed here? Nine CARDINAL chances out of ten CARDINAL we'd both wind up in a concentration camp. Isn't that true, Louis PERSON ? / -- Last night TIME we said a great many things. You said I was to do the thinking for both of us. Well, I've done a lot of it since then and it all adds up to one thing. You're getting on that plane with Victor PERSON where you belong. / I'm staying here with him 'til the plane gets safely away. / Because you're getting on that plane. / Everything is quite all right. / No, not yet. / All right, I will. Here's looking at you, kid. / All except one. He won't have you. / And Laszlo PERSON ? / Huh. But it's still a story without an ending. What about now? / When did you first ORDINAL find out he was alive? / Why weren't you honest with me? Why did you keep your marriage a secret? / And then? / If Laszlo PERSON and the cause mean so much to you, you won't stop at anything. All right, I'll make it easier for you. / No. / I don't have to. I've got them right here. / What of it? I'm going to die in Casablanca GPE . It's a good spot for it. / -- I wouldn't believe you, no matter what you told me. You'd say anything now to get what you want. / -- I wouldn't bring up Paris GPE if I were you. It's poor salesmanship. / I'm not fighting for anything anymore, except myself. I'm the only cause I'm interested in. / Do I have to hear again what a great man your husband is? What an important cause he's fighting for? / I went through all that with your husband. It's no deal. / Your unexpected visit isn't connected by any chance with the letters of transit? It seems that as long as I have those letters I'll never be lonely. / You use " Richard PERSON " again? We're back in Paris GPE . / I told you this morning TIME you'd come around, but this is a little ahead of schedule. Well, won't you sit down? / How did you get in? / All the same, someday you'll lie to Laszlo PERSON . You'll be there. / Well, I'm not running away any more. I'm settled now, above a saloon, it's true, but... walk up a flight. I'll be expecting you. / Did you run out on me because you couldn't take it? Because you knew what it would be like, hiding from the police, running away all the time? / Why not? After all, I got stuck with a railway ticket. I think I'm entitled to know. / Well, you can tell me now. I'm reasonably sober. / Why did you come back? To tell me why you ran out on me at the railway station? / I'm sorry I was in no condition to receive you when you called on me last night TIME . / Yes, that's very pretty. I heard a story once. As a matter of fact, I've heard a lot of stories in my time. They went along with the sound of a tinny piano playing in the parlor downstairs, "Mister, I met a man once when I was a kid," it'd always begin. Huh. I guess neither one of our stories was very funny. Tell me, who was it you left me for? Was it Laszlo, or were there others in between? Or aren't you the kind that tells? / Well, go on, tell it. Maybe one will come to you as you go along. / Has it got a wow finish? / Well, I did. Every one of them. Mostly I remember the last one. A wow finish. A guy standing on a station platform in the rain with a comical look on his face, because his insides had been kicked out. / Huh! You understand how I feel. How long was it we had, honey? / It's funny about your voice, how it hasn't changed. I can still hear it. " Richard PERSON dear, I'll go with you any place. We'll get on a train together and never stop." / Why did you have to come to Casablanca GPE ? There are other places. / Especially tonight TIME . / Oh. I saved my first ORDINAL drink to have with you. Here. / Well, why not? The captain on a ship can. It doesn't seem fair that... Hey, hey, what's wrong, kid? / Yes, I guess it is a little too far ahead. Well, let's see. What about the engineer? Why can't he marry us on the train? / All right. At a quarter to five CARDINAL . Say, why don't we get married in Marseilles GPE ? / -- The train for Marseilles GPE leaves at five o'clock TIME . I'll pick you up at your hotel at four-thirty DATE . / No, no, no, no. We must leave. / I know very little about you, just the fact that you had your teeth straightened. / Ah, that's the new German NORP 77. And judging by the sound, only about thirty-five miles CARDINAL away. / Looking for a job. / Yeah. It's pretty bad timing. Where were you, say, ten years ago DATE ? / My German NORP 's a little rusty. / I'm on their blacklist already, their roll of honor. / I'm sorry for asking. I forgot we said "no questions." / Uh huh. / Why I'm so lucky. Why I should find you waiting for me to come along. / And I was wondering. / In America GPE they'd bring only a penny MONEY . I guess that's about all they're worth. / Here's looking at you, kid. / Who are you really? And what were you before? What did you do and what did you think? Huh? / He hasn't played it in a long time. / I will. / I remember every detail. The Germans NORP wore gray, you wore blue. / Not an easy day to forget. / -- It was " La Belle Aurore WORK_OF_ART ." / Here it is. / But you didn't know she was at my place last night TIME when you were. She came there for the letters of transit. Isn't that true, Ilsa GPE ? / I'm going to anyway, because it may make a difference to you later on. You said you knew about Ilsa GPE and me. / All except one CARDINAL thing. There's something you should know before you leave. / Good. I've got the letters right here, all made out in blank. / Oh, never mind about that. You won't have any trouble in Lisbon GPE , will you? / Keep it. You'll need it in America GPE . / Oh, save it. We've still lots of things to do. / You love her that much? / You seem to know all about my destiny. / Yes, I get the point. / What of it? Then it'll be out of it's misery. / Don't you sometimes wonder if it's worth all this? I mean what you're fighting for? / Had a close one, eh? / Well, this might come in handy. / I said, ask your wife. / There is. I suggest that you ask your wife. / My friend, you could make it a million francs MONEY , or three CARDINAL , my answer would still be the same. / I appreciate it, but I don't accept it. / Yes. I found that a very expensive hobby, too. But then I never was much of a businessman. / What of it? / I'm not interested in politics. The problems of the world are not in my department. I'm a saloon keeper. / My office. / Go ahead. / I take that as a great compliment to Sam PERSON . I suppose he means to you Paris GPE of, well, happier days. / Good evening TIME . / Signor Ferrari PERSON is the fat gent at the table. / Any time. / Not at all. / We all try. You succeed. / Your work. / And I congratulate you. / And about Victor Laszlo PERSON everywhere. / How do you do? / Louis, I think this is the beginning of a beautiful friendship. / Our expenses? / My letter of transit? I could use a trip. But it doesn't make any difference about our bet. You still owe me ten thousand francs MONEY . / Maybe, but it seemed like a good time to start. / As soon as the plane goes, Louis PERSON . / Anyway, thanks for helping me out. / Stay where you are. I don't know what you're talking about. / And the names are Mr. and Mrs. Victor Laszlo PERSON . / If you don't mind, you fill in the names. That will make it even more official. / Just the same, you call the airport and let me hear you tell them. And remember, this gun's pointed right at your heart. / I do. We've got plenty of time to discuss that later. / I have. Sit down over there. / -- Not so fast, Louis PERSON . Nobody's going to be arrested. Not for a while yet. / Sam PERSON 's piano. / I have the letters right here. / Yes, I know what you mean, but I've already spoken to Ferrari NORP . You'll still win at roulette. / I thought I asked you to tie up your watchdogs. / You're late. / Oh, thanks. / I'll make the arrangements right now with Laszlo PERSON in the visitor's pen. / Well, she isn't just any woman. / I'll arrange to have Laszlo PERSON come there to pick up the letters of transit, and that'll give you the criminal grounds on which to make the arrest. You get him, and we get away. To the Germans NORP that last will be just a minor annoyance. / Then release him. You be at my place a half hour TIME before the plane leaves. / Ilsa is Laszlo PERSON 's wife. She probably knows things that Strasser PERSON would like to know. Louis PERSON , I'll make a deal with you. Instead of this petty charge you have against him, you can get something really big, something that would chuck PERSON him in a concentration camp for years DATE . That would be quite a feather in your cap, wouldn't it? / Ilsa Lund. That ought to put your mind to rest about my helping Laszlo PERSON escape. The last man I want to see in America GPE . / And I'm taking a friend with me. One CARDINAL you'll appreciate. / You're not very subtle, but you are effective. I, I get the point. Yes, I have the letters, but I intend using them myself. I'm leaving Casablanca PERSON on tonight TIME 's plane, the last plane. / Well, they're all excellent reasons. / -- What makes you think I'd stick my neck out for Laszlo PERSON ? / But you haven't any actual proof, and you know it. This isn't Germany GPE or occupied France GPE . All you can do is fine him a few thousand francs MONEY and give him thirty days DATE . You might as well let him go now. / How can you close me up? On what grounds? / Put it down as a gesture to love. / Yeah? Why? / So Yvonne PERSON 's gone over to the enemy. / Well, it looks like you're a little late. / Louis GPE , are you pro-Vichy or Free French NORP ? / That was some going-over your men gave my place this afternoon TIME . We just barely got cleaned up in time to open. / Oh, it's my party. / -- Thanks. I will. / -- Hello, Ilsa GPE . / I was born in New York City GPE if that'll help you any. / Oh, how do you do? / Yeah, you were saying? / Oh, I see. Gestapo PERSON spank. / Maybe. Well, it seems you are determined to keep Laszlo PERSON here. / And got well paid for it on both occasions. / Louis, whatever gave you the impression that I might be interested in helping Laszlo PERSON escape? / Maybe he's not quite as romantic as you are. / He'll take one. / Why two CARDINAL ? / Okay. / I just paid out twenty CARDINAL . I'd like to get it back. / Twenty thousand francs MONEY says it isn't. / -- Stop it. He escaped from a concentration camp and the Nazis NORP have been chasing him all over Europe LOC . / His escape. / It'll be interesting to see how he manages. / Well, he's succeeded in impressing half CARDINAL the world. / Victor Laszlo PERSON ? / Yeah? What's his name? / I thought it was because we let you win at roulette. / Yeah? Have a brandy? / I see. And what's Strasser PERSON doing here? He certainly didn't come all the way to Casablanca GPE to witness a demonstration of your efficiency. / Our entertainment is enough. / I stick my neck out for nobody. / What, again? / I was misinformed. / My health. I came to Casablanca GPE for the waters. / It was a combination of all three CARDINAL . / Why? What's in Lisbon GPE ? / When it comes to women, you're a true democrat NORP . / Hello, Louis PERSON . / Come on, we're going to get your coat. / -- Sacha, call a cab. / Sacha, she's had enough. / I never make plans that far ahead. / That's so long ago, I don't remember. / Put it down! / Put that phone down! / I was willing to shoot Captain Renault PERSON , and I'm willing to shoot you. / You'll excuse me, gentlemen. Your business is politics. Mine is running a saloon. / Not particularly. I understand the point of view of the hound, too. / My interest in whether Victor Laszlo PERSON stays or goes is purely a sporting one. / Are my eyes really brown? / I haven't the slightest idea. / Well, there are certain sections of New York GPE , Major ORG , that I wouldn't advise you to try to invade. / It's not particularly my beloved Paris GPE . / There seems to be no secret about that. / I'm a drunkard. / Make it official, if you like. / Don't be a fool. You can't get away. / I don't want them here overnight TIME . / For how long? / Who did you bribe for your visa? Renault ORG or yourself? / I don't mind a parasite. I object to a cut-rate one. / For a price, Ugarte ORG , for a price. / If I gave you any thought, I probably would. / No. / They got a lucky break. Yesterday DATE they were just two CARDINAL German NORP clerks. Today DATE they're the 'Honored Dead'. / -- You thought what? / Well, what makes you think I haven't? / Hello Ugarte ORG . / Where is she? Have you seen her? / And getting closer every minute TIME . Here. Drink up. We'll never finish the other three CARDINAL . / You said it! Here's looking at you, kid. / If she can stand it, I can. Play it! / You played it for her and you can play it for me. / Well, stop it. You know what I want to hear. / What's that you're playing? / I bet they're asleep in New York GPE . I'll bet they're asleep all over America GPE . / Sam PERSON , if it's December 1941 DATE in Casablanca GPE , what time is it in New York GPE ? / They grab Ugarte ORG and she walks in. Well, that's the way it goes. One CARDINAL in, one CARDINAL out. Sam PERSON ? / Shut up and go home, will you? / She's coming back. I know she's coming back. / No, sir. I'm waiting for a lady. / Then don't have a drink. / Good. Then have a drink. / No. / No. / Not right now. / Yeah? / Sorry. / He'll double what I pay you. / Sam PERSON , Ferrari NORP wants you to work for him at the Blue Parrot FAC .

Oh I can't remember it, Miss Ilsa PERSON . I'm a little rusty on it. / I don't know what you mean, Miss Ilsa PERSON . / Leave him alone, Miss Ilsa PERSON . You're bad luck to him. / Oh, he never... well... he's got a girl up at the Blue Parrot PRODUCT . He goes up there all the time. / Not tonight TIME no more. He ain't coming. Uh, he went home. / I don't know. I ain't seen him all night. / Yes, ma'am. / Yes, ma'am. A lot of water under the bridge. / Hello, Miss Ilsa PERSON . I never expected to see you again. / No, Mr. Richard PERSON . I can't find her. / The Germans'll be here pretty soon now, and they'll come looking for you. And don't forget there's a price on your head. / This sort of takes the sting out of being occupied, doesn't it, Mr. Richard PERSON ? / Yes, boss. / Well, I don't think I can remember it. / No, I don't. / Just a little something of my own. / Uh, my watch stopped. / Yeah, boss? / No, sir. I'm staying right here. / We'll take the car and drive all night TIME . We'll get drunk. We'll go fishing and stay away until she's gone. / Please, boss, let's go. Ain't nothing but trouble for you here. / Boss, let's get out of here. / No. Not me, boss. / Well, I ain't sleepy either. / You ever going to bed? / Ain't you planning on going to bed in the near future? / Boss, ain't you going to bed? / Boss! / Yeah, but I ain't got time to spend the money I make here. / I like it fine here.

My dear Mademoiselle ORG , perhaps you have already observed that in Casablanca GPE , human life is cheap. Good night, Mademoiselle ORG . / It is possible the French NORP authorities will find a reason to put him in the concentration camp here. / There are only two CARDINAL other alternatives for him. / Uh huh. Under a safe conduct from me. / That is also true, except for one CARDINAL destination, to return to occupied France GPE . / Mademoiselle, after this disturbance it is not safe for Laszlo PERSON to stay in Casablanca GPE . / And Mademoiselle PERSON ? / For the time being. / Herr Laszlo PERSON , you have a reputation for eloquence which I can now understand. But in one respect you are mistaken. You said the enemies of the Reich PERSON could all be replaced, but there is one CARDINAL exception. No one could take your place in the event anything unfortunate should occur to you while you were trying to escape. / You will give us the names? / Yes, even in Berlin GPE . If you will furnish me with their names and their exact whereabouts, you will have your visa in the morning TIME . / You know the leaders of the underground movement in Paris GPE , in Prague GPE , in Brussels GPE , in Amsterdam GPE , in Oslo GPE , in Belgrade GPE , in Athens GPE . / Don't be in such a hurry. You have all the time in the world. You may be in Casablanca GPE indefinitely... or you may leave for Lisbon GPE tomorrow DATE , on one CARDINAL condition. / Not at all. Captain Renault PERSON 's signature is necessary on every exit visa. Captain, would you think it is possible that Herr Laszlo PERSON will receive a visa? / Very well, Herr Laszlo PERSON , we will not mince words. You are an escaped prisoner of the Reich PERSON . So far you have been fortunate enough in eluding us. You have reached Casablanca WORK_OF_ART . It is my duty to see that you stay in Casablanca GPE . / Then we shall state another time and another place. Tomorrow DATE at ten CARDINAL in the Prefect FAC 's office, with Mademoiselle ORG . / I should like to discuss some matters arising from your presence on French NORP soil. / You were a Czechoslovakian NORP . Now you are a subject of the German Reich ORG ! / Why do you stand here? Why don't you stop him? / What is the meaning of that phone call? / Find one. / Yes, much too good a time. The place is to be closed. / You see what I mean? If Laszlo PERSON 's presence in a cafe can inspire this unfortunate demonstration, what more will his presence in Casablanca GPE bring on? I advise that this place be shut up at once. / Uh, huh. I have been thinking. It is too dangerous if we let him go. It may be too dangerous if we let him stay. / We are concerned about more than Casablanca GPE . We know that every French NORP province in Africa LOC is honeycombed with traitors waiting for their chance, waiting, perhaps, for a leader. / And if it should change? / Captain Renault PERSON , are you entirely certain which side you're on? / You see, Captain PERSON , the situation is not as much under control as you believe. / As to Laszlo PERSON , we want him watched twenty-four hours TIME a day. / You give him credit for too much cleverness. My impression was that he's just another blundering American NORP . / I strongly suspect that Ugarte ORG left the letters of transit with Mr. Blaine PERSON . I would suggest you search the cafe immediately and thoroughly. / Mademoiselle. / Perhaps. / I admit he is very clever. Three CARDINAL times he slipped through our fingers. In Paris GPE he continued his activities. We intend not to let it happen again. / You weren't always so carefully neutral. We have a complete dossier on you. / How about New York GPE ? / Do you mind if I ask you a few questions? Unofficially, of course. / You repeat " Third Reich WORK_OF_ART " as though you expected there to be others. / How do you do, Mr. Rick PERSON ? / I expected no less, Captain. / Thank you. / Champagne and a tin of caviar. / Good evening TIME , Captain PERSON . / I have already heard about this cafe, and also about Mr. Rick PERSON himself. / By the way, the murder of the couriers, what has been done? / Oh, we Germans NORP must get used to all climates, from Russia GPE to the Sahara LOC . But perhaps you were not referring to the weather. / Thank you, Captain. It's very good to be here. / Get me the Radio Tower ORG ! / Hello? / I would advise you not to interfere. / Good evening TIME , Mr. Blaine PERSON . / Victor Laszlo PERSON published the foulest lies in the Prague GPE newspapers until the very day DATE we marched in, and even after that he continued to print scandal sheets in a cellar. / In this case, you have no sympathy for the fox, huh? / You will forgive my curiosity, Mr. Blaine PERSON . The point is, an enemy of the Reich PERSON has come to Casablanca GPE and we are checking up on anybody who can be of any help to us. / Aha. Who do you think will win the war? / Are you one of those people who cannot imagine the Germans NORP in their beloved Paris GPE ? / I understand you came here from Paris GPE at the time of the occupation. / What is your nationality?

Rick PERSON , hide me. Do something! You must help me, Rick PERSON . Do something! / Rick PERSON ! Rick PERSON , help me! / Don't be afraid of that. Please keep them for me. Thank you. I knew I could trust you. / Perhaps an hour TIME , perhaps a little longer. / One CARDINAL moment. Tonight TIME I'll be selling those for more money than even I have ever dreamed of, and then, addio Casablanca PERSON ! You know, Rick PERSON , I have many friends in Casablanca GPE , but somehow, just because you despise me you're the only one I trust. Will you keep these for me? Please. / Myself. I found myself much more reasonable. / Well, Rick PERSON , after tonight TIME I'll be through with the whole business, and I am leaving finally this Casablanca GPE . / But think of all the poor devils who cannot meet Renault ORG 's price. I get it for them for half CARDINAL . Is that so parasitic? / But why? Oh, you object to the kind of business I do, huh? But think of all those poor refugees who must rot in this place if I didn't help them. That's not so bad. Through ways of my own I provide them with exit visas. / I forgot. You never drink with... I'll have another, please. You despise me, don't you? / Thank you. Will you have a drink with me please? / You are a very cynical person, Rick PERSON , if you'll forgive me for saying so. / May I? Too bad about those two CARDINAL German NORP couriers, wasn't it? / Oh, nothing. But when you first ORDINAL came to Casablanca GPE , I thought -- / Huh. You know, Rick PERSON , watching you just now with the Deutsches Bank ORG , one would think you'd been doing this all your life. / Uh, excuse me, please. Hello, Rick PERSON .

# Define a function to find keywords in its context
def find_sentences_with_keyword(keyword, document): 
    # loop through all the sentences in the document and pull out the text of each sentence
    for sentence in document.sents:
        sentence = sentence.text        
        # check to see if the keyword is in the sentence (and ignore capitalization by making both lowercase)
        if keyword.lower() in sentence.lower():            
            # use regular expression to replace linebreaks and to make the keyword bolded, again ignoring capitalization
            sentence = re.sub('\n', ' ', sentence)
            sentence = re.sub(f"{keyword}", f"**{keyword}**", sentence, flags=re.IGNORECASE)            
            display(Markdown(sentence))

# Highlight the name of the protagonist Rick in its context of the lines of the second character of the movie
find_sentences_with_keyword(keyword="Rick", document=nlp(movie_casablanca['lines'][1]))

Rick's wouldn't be Rick's without them.

/ Rick, don't be a fool.

Rick, I'll put my cards on the table.

My dear Rick, when will you realize that in this world today isolationism is no longer a practical policy?

Hello, Rick. /

He is a difficult customer, that Rick.

Not for sure, Monsieur, but I will venture to guess that Ugarte left those letters with Monsieur Rick.

# Create a list of tokens and POS labels from document if the token is a word 
tokens_and_labels = [(token.text, token.pos_) for token in nlp(movie_casablanca['lines'][1]) if token.is_alpha]
tokens_and_labels

[('I', 'PRON'),
 ('shall', 'AUX'),
 ('remember', 'VERB'),
 ('to', 'PART'),
 ('pay', 'VERB'),
 ('it', 'PRON'),
 ('to', 'ADP'),
 ('myself', 'PRON'),
 ('Of', 'ADV'),
 ('course', 'ADV'),
 ('they', 'PRON'),
 ('stay', 'VERB'),
 ('Rick', 'PROPN'),
 ('would', 'AUX'),
 ('be', 'AUX'),
 ('Rick', 'PROPN'),
 ('without', 'ADP'),
 ('them', 'PRON'),
 ('Hmmm', 'PROPN'),
 ('I', 'PRON'),
 ('happen', 'VERB'),
 ('to', 'PART'),
 ('know', 'VERB'),
 ('that', 'SCONJ'),
 ('he', 'PRON'),
 ('gets', 'VERB'),
 ('ten', 'NUM'),
 ('percent', 'NOUN'),
 ('But', 'CCONJ'),
 ('he', 'PRON'),
 ('worth', 'ADJ'),
 ('five', 'NUM'),
 ('Ah', 'INTJ'),
 ('to', 'PART'),
 ('get', 'VERB'),
 ('out', 'ADP'),
 ('of', 'ADP'),
 ('Casablanca', 'PROPN'),
 ('and', 'CCONJ'),
 ('go', 'VERB'),
 ('to', 'ADP'),
 ('America', 'PROPN'),
 ('You', 'PRON'),
 ('a', 'DET'),
 ('lucky', 'ADJ'),
 ('man', 'NOUN'),
 ('Shall', 'AUX'),
 ('we', 'PRON'),
 ('draw', 'VERB'),
 ('up', 'ADP'),
 ('the', 'DET'),
 ('papers', 'NOUN'),
 ('or', 'CCONJ'),
 ('is', 'AUX'),
 ('our', 'PRON'),
 ('handshake', 'NOUN'),
 ('good', 'ADJ'),
 ('enough', 'ADV'),
 ('Rick', 'PROPN'),
 ('do', 'AUX'),
 ('be', 'AUX'),
 ('a', 'DET'),
 ('fool', 'NOUN'),
 ('Take', 'VERB'),
 ('me', 'PRON'),
 ('into', 'ADP'),
 ('your', 'PRON'),
 ('confidence', 'NOUN'),
 ('You', 'PRON'),
 ('need', 'VERB'),
 ('a', 'DET'),
 ('partner', 'NOUN'),
 ('Rick', 'PROPN'),
 ('I', 'PRON'),
 ('put', 'VERB'),
 ('my', 'PRON'),
 ('cards', 'NOUN'),
 ('on', 'ADP'),
 ('the', 'DET'),
 ('table', 'NOUN'),
 ('I', 'PRON'),
 ('think', 'VERB'),
 ('you', 'PRON'),
 ('know', 'VERB'),
 ('where', 'SCONJ'),
 ('those', 'DET'),
 ('letters', 'NOUN'),
 ('are', 'AUX'),
 ('Naturally', 'ADV'),
 ('there', 'PRON'),
 ('will', 'AUX'),
 ('be', 'AUX'),
 ('a', 'DET'),
 ('few', 'ADJ'),
 ('incidental', 'ADJ'),
 ('expenses', 'NOUN'),
 ('That', 'PRON'),
 ('is', 'AUX'),
 ('the', 'DET'),
 ('proposition', 'NOUN'),
 ('I', 'PRON'),
 ('have', 'VERB'),
 ('for', 'ADP'),
 ('whoever', 'PRON'),
 ('has', 'AUX'),
 ('those', 'DET'),
 ('letters', 'NOUN'),
 ('I', 'PRON'),
 ('have', 'VERB'),
 ('a', 'DET'),
 ('proposition', 'NOUN'),
 ('for', 'ADP'),
 ('whoever', 'PRON'),
 ('has', 'AUX'),
 ('those', 'DET'),
 ('letters', 'NOUN'),
 ('I', 'PRON'),
 ('will', 'AUX'),
 ('handle', 'VERB'),
 ('the', 'DET'),
 ('entire', 'ADJ'),
 ('transaction', 'NOUN'),
 ('get', 'VERB'),
 ('rid', 'VERB'),
 ('of', 'ADP'),
 ('the', 'DET'),
 ('letters', 'NOUN'),
 ('take', 'VERB'),
 ('all', 'DET'),
 ('the', 'DET'),
 ('risk', 'NOUN'),
 ('for', 'ADP'),
 ('a', 'DET'),
 ('small', 'ADJ'),
 ('percentage', 'NOUN'),
 ('If', 'SCONJ'),
 ('I', 'PRON'),
 ('could', 'AUX'),
 ('lay', 'VERB'),
 ('my', 'PRON'),
 ('hands', 'NOUN'),
 ('on', 'ADP'),
 ('those', 'DET'),
 ('letters', 'NOUN'),
 ('I', 'PRON'),
 ('could', 'AUX'),
 ('make', 'VERB'),
 ('a', 'DET'),
 ('fortune', 'NOUN'),
 ('Of', 'ADV'),
 ('course', 'ADV'),
 ('not', 'PART'),
 ('What', 'PRON'),
 ('upsets', 'VERB'),
 ('me', 'PRON'),
 ('is', 'AUX'),
 ('the', 'DET'),
 ('fact', 'NOUN'),
 ('that', 'SCONJ'),
 ('Ugarte', 'PROPN'),
 ('is', 'AUX'),
 ('dead', 'ADJ'),
 ('and', 'CCONJ'),
 ('no', 'DET'),
 ('one', 'NOUN'),
 ('knows', 'VERB'),
 ('where', 'SCONJ'),
 ('those', 'DET'),
 ('letters', 'NOUN'),
 ('of', 'ADP'),
 ('transit', 'NOUN'),
 ('are', 'AUX'),
 ('The', 'DET'),
 ('bourbon', 'NOUN'),
 ('The', 'DET'),
 ('news', 'NOUN'),
 ('about', 'ADP'),
 ('Ugarte', 'PROPN'),
 ('upset', 'VERB'),
 ('me', 'PRON'),
 ('very', 'ADV'),
 ('much', 'ADV'),
 ('Carrying', 'VERB'),
 ('charges', 'NOUN'),
 ('my', 'PRON'),
 ('boy', 'NOUN'),
 ('carrying', 'VERB'),
 ('charges', 'NOUN'),
 ('Here', 'ADV'),
 ('sit', 'VERB'),
 ('down', 'ADP'),
 ('There', 'PRON'),
 ('something', 'PRON'),
 ('I', 'PRON'),
 ('want', 'VERB'),
 ('to', 'PART'),
 ('talk', 'VERB'),
 ('over', 'ADP'),
 ('with', 'ADP'),
 ('you', 'PRON'),
 ('anyhow', 'ADV'),
 ('No', 'DET'),
 ('hurry', 'NOUN'),
 ('I', 'PRON'),
 ('have', 'VERB'),
 ('it', 'PRON'),
 ('sent', 'VERB'),
 ('over', 'ADP'),
 ('Have', 'VERB'),
 ('a', 'DET'),
 ('drink', 'NOUN'),
 ('with', 'ADP'),
 ('me', 'PRON'),
 ('My', 'PRON'),
 ('dear', 'ADJ'),
 ('Rick', 'PROPN'),
 ('when', 'SCONJ'),
 ('will', 'AUX'),
 ('you', 'PRON'),
 ('realize', 'VERB'),
 ('that', 'SCONJ'),
 ('in', 'ADP'),
 ('this', 'DET'),
 ('world', 'NOUN'),
 ('today', 'NOUN'),
 ('isolationism', 'NOUN'),
 ('is', 'AUX'),
 ('no', 'ADV'),
 ('longer', 'ADV'),
 ('a', 'DET'),
 ('practical', 'ADJ'),
 ('policy', 'NOUN'),
 ('Suppose', 'VERB'),
 ('we', 'PRON'),
 ('ask', 'VERB'),
 ('Sam', 'PROPN'),
 ('Maybe', 'ADV'),
 ('he', 'PRON'),
 ('like', 'VERB'),
 ('to', 'PART'),
 ('make', 'VERB'),
 ('a', 'DET'),
 ('change', 'NOUN'),
 ('That', 'PRON'),
 ('too', 'ADV'),
 ('bad', 'ADJ'),
 ('That', 'PRON'),
 ('Casablanca', 'PROPN'),
 ('leading', 'VERB'),
 ('commodity', 'NOUN'),
 ('In', 'ADP'),
 ('refugees', 'NOUN'),
 ('alone', 'ADV'),
 ('we', 'PRON'),
 ('could', 'AUX'),
 ('make', 'VERB'),
 ('a', 'DET'),
 ('fortune', 'NOUN'),
 ('if', 'SCONJ'),
 ('you', 'PRON'),
 ('would', 'AUX'),
 ('work', 'VERB'),
 ('with', 'ADP'),
 ('me', 'PRON'),
 ('through', 'ADP'),
 ('the', 'DET'),
 ('black', 'ADJ'),
 ('market', 'NOUN'),
 ('What', 'PRON'),
 ('do', 'AUX'),
 ('you', 'PRON'),
 ('want', 'VERB'),
 ('for', 'ADP'),
 ('Sam', 'PROPN'),
 ('You', 'PRON'),
 ('have', 'AUX'),
 ('heard', 'VERB'),
 ('my', 'PRON'),
 ('offer', 'NOUN'),
 ('Fine', 'INTJ'),
 ('but', 'CCONJ'),
 ('I', 'PRON'),
 ('would', 'AUX'),
 ('like', 'VERB'),
 ('to', 'PART'),
 ('buy', 'VERB'),
 ('your', 'PRON'),
 ('cafe', 'NOUN'),
 ('Hello', 'PROPN'),
 ('Rick', 'PROPN'),
 ('It', 'PRON'),
 ('was', 'AUX'),
 ('gracious', 'ADJ'),
 ('of', 'ADP'),
 ('you', 'PRON'),
 ('to', 'PART'),
 ('share', 'VERB'),
 ('it', 'PRON'),
 ('with', 'ADP'),
 ('me', 'PRON'),
 ('Good', 'ADJ'),
 ('day', 'NOUN'),
 ('Mademoiselle', 'PROPN'),
 ('Monsieur', 'PROPN'),
 ('He', 'PRON'),
 ('is', 'AUX'),
 ('a', 'DET'),
 ('difficult', 'ADJ'),
 ('customer', 'NOUN'),
 ('that', 'SCONJ'),
 ('Rick', 'PROPN'),
 ('One', 'NUM'),
 ('never', 'ADV'),
 ('knows', 'VERB'),
 ('what', 'PRON'),
 ('he', 'PRON'),
 ('do', 'VERB'),
 ('or', 'CCONJ'),
 ('why', 'SCONJ'),
 ('But', 'CCONJ'),
 ('it', 'PRON'),
 ('is', 'AUX'),
 ('worth', 'ADJ'),
 ('a', 'DET'),
 ('chance', 'NOUN'),
 ('Not', 'PART'),
 ('for', 'ADP'),
 ('sure', 'ADJ'),
 ('Monsieur', 'PROPN'),
 ('but', 'CCONJ'),
 ('I', 'PRON'),
 ('will', 'AUX'),
 ('venture', 'VERB'),
 ('to', 'PART'),
 ('guess', 'VERB'),
 ('that', 'SCONJ'),
 ('Ugarte', 'PROPN'),
 ('left', 'VERB'),
 ('those', 'DET'),
 ('letters', 'NOUN'),
 ('with', 'ADP'),
 ('Monsieur', 'PROPN'),
 ('Rick', 'PROPN'),
 ('Those', 'DET'),
 ('letters', 'NOUN'),
 ('were', 'AUX'),
 ('not', 'PART'),
 ('found', 'VERB'),
 ('on', 'ADP'),
 ('Ugarte', 'PROPN'),
 ('when', 'SCONJ'),
 ('they', 'PRON'),
 ('arrested', 'VERB'),
 ('him', 'PRON'),
 ('I', 'PRON'),
 ('observe', 'VERB'),
 ('that', 'SCONJ'),
 ('you', 'PRON'),
 ('in', 'ADP'),
 ('one', 'NUM'),
 ('respect', 'NOUN'),
 ('are', 'AUX'),
 ('a', 'DET'),
 ('very', 'ADV'),
 ('fortunate', 'ADJ'),
 ('man', 'NOUN'),
 ('Monsieur', 'PROPN'),
 ('I', 'PRON'),
 ('am', 'AUX'),
 ('moved', 'VERB'),
 ('to', 'PART'),
 ('make', 'VERB'),
 ('one', 'NUM'),
 ('more', 'ADJ'),
 ('suggestion', 'NOUN'),
 ('why', 'SCONJ'),
 ('I', 'PRON'),
 ('do', 'AUX'),
 ('not', 'PART'),
 ('know', 'VERB'),
 ('because', 'SCONJ'),
 ('it', 'PRON'),
 ('can', 'AUX'),
 ('not', 'PART'),
 ('possibly', 'ADV'),
 ('profit', 'VERB'),
 ('me', 'PRON'),
 ('but', 'CCONJ'),
 ('have', 'AUX'),
 ('you', 'PRON'),
 ('heard', 'VERB'),
 ('about', 'ADP'),
 ('Signor', 'PROPN'),
 ('Ugarte', 'PROPN'),
 ('and', 'CCONJ'),
 ('the', 'DET'),
 ('letters', 'NOUN'),
 ('of', 'ADP'),
 ('transit', 'NOUN'),
 ('Well', 'INTJ'),
 ('good', 'ADJ'),
 ('luck', 'NOUN'),
 ('But', 'CCONJ'),
 ('be', 'AUX'),
 ('careful', 'ADJ'),
 ('You', 'PRON'),
 ('know', 'VERB'),
 ('you', 'PRON'),
 ('being', 'AUX'),
 ('shadowed', 'VERB'),
 ('We', 'PRON'),
 ('might', 'AUX'),
 ('as', 'ADV'),
 ('well', 'ADV'),
 ('be', 'AUX'),
 ('frank', 'ADJ'),
 ('Monsieur', 'PROPN'),
 ('It', 'PRON'),
 ('will', 'AUX'),
 ('take', 'VERB'),
 ('a', 'DET'),
 ('miracle', 'NOUN'),
 ('to', 'PART'),
 ('get', 'VERB'),
 ('you', 'PRON'),
 ('out', 'ADP'),
 ('of', 'ADP'),
 ('Casablanca', 'PROPN'),
 ('And', 'CCONJ'),
 ('the', 'DET'),
 ('Germans', 'PROPN'),
 ('have', 'AUX'),
 ('outlawed', 'VERB'),
 ('miracles', 'NOUN'),
 ('As', 'ADP'),
 ('leader', 'NOUN'),
 ('of', 'ADP'),
 ('all', 'DET'),
 ('illegal', 'ADJ'),
 ('activities', 'NOUN'),
 ('in', 'ADP'),
 ('Casablanca', 'PROPN'),
 ('I', 'PRON'),
 ('am', 'AUX'),
 ('an', 'DET'),
 ('influential', 'ADJ'),
 ('and', 'CCONJ'),
 ('respected', 'ADJ'),
 ('man', 'NOUN'),
 ('It', 'PRON'),
 ('would', 'AUX'),
 ('not', 'PART'),
 ('be', 'AUX'),
 ('worth', 'ADJ'),
 ('my', 'PRON'),
 ('life', 'NOUN'),
 ('to', 'PART'),
 ('do', 'VERB'),
 ('anything', 'PRON'),
 ('for', 'ADP'),
 ('Monsieur', 'PROPN'),
 ('Laszlo', 'PROPN'),
 ('You', 'PRON'),
 ('however', 'ADV'),
 ('are', 'AUX'),
 ('a', 'DET'),
 ('different', 'ADJ'),
 ('matter', 'NOUN')]

# Define a function to get all two-word combinations
def get_bigrams(word_list, number_consecutive_words=2):  
    ngrams = []
    adj_length_of_word_list = len(word_list) - (number_consecutive_words - 1)   
    # loop through numbers from 0 to the (slightly adjusted) length of your word list
    for word_index in range(adj_length_of_word_list):       
        # index the list at each number, grabbing the word at that number index as well as N number of words after it
        ngram = word_list[word_index : word_index + number_consecutive_words]        
        # append this word combo to the master list "ngrams"
        ngrams.append(ngram)        
    return ngrams

# Getting all bigram of the lines, including both the word and its POS label
bigrams = get_bigrams(tokens_and_labels)
bigrams

[[('I', 'PRON'), ('shall', 'AUX')],
 [('shall', 'AUX'), ('remember', 'VERB')],
 [('remember', 'VERB'), ('to', 'PART')],
 [('to', 'PART'), ('pay', 'VERB')],
 [('pay', 'VERB'), ('it', 'PRON')],
 [('it', 'PRON'), ('to', 'ADP')],
 [('to', 'ADP'), ('myself', 'PRON')],
 [('myself', 'PRON'), ('Of', 'ADV')],
 [('Of', 'ADV'), ('course', 'ADV')],
 [('course', 'ADV'), ('they', 'PRON')],
 [('they', 'PRON'), ('stay', 'VERB')],
 [('stay', 'VERB'), ('Rick', 'PROPN')],
 [('Rick', 'PROPN'), ('would', 'AUX')],
 [('would', 'AUX'), ('be', 'AUX')],
 [('be', 'AUX'), ('Rick', 'PROPN')],
 [('Rick', 'PROPN'), ('without', 'ADP')],
 [('without', 'ADP'), ('them', 'PRON')],
 [('them', 'PRON'), ('Hmmm', 'PROPN')],
 [('Hmmm', 'PROPN'), ('I', 'PRON')],
 [('I', 'PRON'), ('happen', 'VERB')],
 [('happen', 'VERB'), ('to', 'PART')],
 [('to', 'PART'), ('know', 'VERB')],
 [('know', 'VERB'), ('that', 'SCONJ')],
 [('that', 'SCONJ'), ('he', 'PRON')],
 [('he', 'PRON'), ('gets', 'VERB')],
 [('gets', 'VERB'), ('ten', 'NUM')],
 [('ten', 'NUM'), ('percent', 'NOUN')],
 [('percent', 'NOUN'), ('But', 'CCONJ')],
 [('But', 'CCONJ'), ('he', 'PRON')],
 [('he', 'PRON'), ('worth', 'ADJ')],
 [('worth', 'ADJ'), ('five', 'NUM')],
 [('five', 'NUM'), ('Ah', 'INTJ')],
 [('Ah', 'INTJ'), ('to', 'PART')],
 [('to', 'PART'), ('get', 'VERB')],
 [('get', 'VERB'), ('out', 'ADP')],
 [('out', 'ADP'), ('of', 'ADP')],
 [('of', 'ADP'), ('Casablanca', 'PROPN')],
 [('Casablanca', 'PROPN'), ('and', 'CCONJ')],
 [('and', 'CCONJ'), ('go', 'VERB')],
 [('go', 'VERB'), ('to', 'ADP')],
 [('to', 'ADP'), ('America', 'PROPN')],
 [('America', 'PROPN'), ('You', 'PRON')],
 [('You', 'PRON'), ('a', 'DET')],
 [('a', 'DET'), ('lucky', 'ADJ')],
 [('lucky', 'ADJ'), ('man', 'NOUN')],
 [('man', 'NOUN'), ('Shall', 'AUX')],
 [('Shall', 'AUX'), ('we', 'PRON')],
 [('we', 'PRON'), ('draw', 'VERB')],
 [('draw', 'VERB'), ('up', 'ADP')],
 [('up', 'ADP'), ('the', 'DET')],
 [('the', 'DET'), ('papers', 'NOUN')],
 [('papers', 'NOUN'), ('or', 'CCONJ')],
 [('or', 'CCONJ'), ('is', 'AUX')],
 [('is', 'AUX'), ('our', 'PRON')],
 [('our', 'PRON'), ('handshake', 'NOUN')],
 [('handshake', 'NOUN'), ('good', 'ADJ')],
 [('good', 'ADJ'), ('enough', 'ADV')],
 [('enough', 'ADV'), ('Rick', 'PROPN')],
 [('Rick', 'PROPN'), ('do', 'AUX')],
 [('do', 'AUX'), ('be', 'AUX')],
 [('be', 'AUX'), ('a', 'DET')],
 [('a', 'DET'), ('fool', 'NOUN')],
 [('fool', 'NOUN'), ('Take', 'VERB')],
 [('Take', 'VERB'), ('me', 'PRON')],
 [('me', 'PRON'), ('into', 'ADP')],
 [('into', 'ADP'), ('your', 'PRON')],
 [('your', 'PRON'), ('confidence', 'NOUN')],
 [('confidence', 'NOUN'), ('You', 'PRON')],
 [('You', 'PRON'), ('need', 'VERB')],
 [('need', 'VERB'), ('a', 'DET')],
 [('a', 'DET'), ('partner', 'NOUN')],
 [('partner', 'NOUN'), ('Rick', 'PROPN')],
 [('Rick', 'PROPN'), ('I', 'PRON')],
 [('I', 'PRON'), ('put', 'VERB')],
 [('put', 'VERB'), ('my', 'PRON')],
 [('my', 'PRON'), ('cards', 'NOUN')],
 [('cards', 'NOUN'), ('on', 'ADP')],
 [('on', 'ADP'), ('the', 'DET')],
 [('the', 'DET'), ('table', 'NOUN')],
 [('table', 'NOUN'), ('I', 'PRON')],
 [('I', 'PRON'), ('think', 'VERB')],
 [('think', 'VERB'), ('you', 'PRON')],
 [('you', 'PRON'), ('know', 'VERB')],
 [('know', 'VERB'), ('where', 'SCONJ')],
 [('where', 'SCONJ'), ('those', 'DET')],
 [('those', 'DET'), ('letters', 'NOUN')],
 [('letters', 'NOUN'), ('are', 'AUX')],
 [('are', 'AUX'), ('Naturally', 'ADV')],
 [('Naturally', 'ADV'), ('there', 'PRON')],
 [('there', 'PRON'), ('will', 'AUX')],
 [('will', 'AUX'), ('be', 'AUX')],
 [('be', 'AUX'), ('a', 'DET')],
 [('a', 'DET'), ('few', 'ADJ')],
 [('few', 'ADJ'), ('incidental', 'ADJ')],
 [('incidental', 'ADJ'), ('expenses', 'NOUN')],
 [('expenses', 'NOUN'), ('That', 'PRON')],
 [('That', 'PRON'), ('is', 'AUX')],
 [('is', 'AUX'), ('the', 'DET')],
 [('the', 'DET'), ('proposition', 'NOUN')],
 [('proposition', 'NOUN'), ('I', 'PRON')],
 [('I', 'PRON'), ('have', 'VERB')],
 [('have', 'VERB'), ('for', 'ADP')],
 [('for', 'ADP'), ('whoever', 'PRON')],
 [('whoever', 'PRON'), ('has', 'AUX')],
 [('has', 'AUX'), ('those', 'DET')],
 [('those', 'DET'), ('letters', 'NOUN')],
 [('letters', 'NOUN'), ('I', 'PRON')],
 [('I', 'PRON'), ('have', 'VERB')],
 [('have', 'VERB'), ('a', 'DET')],
 [('a', 'DET'), ('proposition', 'NOUN')],
 [('proposition', 'NOUN'), ('for', 'ADP')],
 [('for', 'ADP'), ('whoever', 'PRON')],
 [('whoever', 'PRON'), ('has', 'AUX')],
 [('has', 'AUX'), ('those', 'DET')],
 [('those', 'DET'), ('letters', 'NOUN')],
 [('letters', 'NOUN'), ('I', 'PRON')],
 [('I', 'PRON'), ('will', 'AUX')],
 [('will', 'AUX'), ('handle', 'VERB')],
 [('handle', 'VERB'), ('the', 'DET')],
 [('the', 'DET'), ('entire', 'ADJ')],
 [('entire', 'ADJ'), ('transaction', 'NOUN')],
 [('transaction', 'NOUN'), ('get', 'VERB')],
 [('get', 'VERB'), ('rid', 'VERB')],
 [('rid', 'VERB'), ('of', 'ADP')],
 [('of', 'ADP'), ('the', 'DET')],
 [('the', 'DET'), ('letters', 'NOUN')],
 [('letters', 'NOUN'), ('take', 'VERB')],
 [('take', 'VERB'), ('all', 'DET')],
 [('all', 'DET'), ('the', 'DET')],
 [('the', 'DET'), ('risk', 'NOUN')],
 [('risk', 'NOUN'), ('for', 'ADP')],
 [('for', 'ADP'), ('a', 'DET')],
 [('a', 'DET'), ('small', 'ADJ')],
 [('small', 'ADJ'), ('percentage', 'NOUN')],
 [('percentage', 'NOUN'), ('If', 'SCONJ')],
 [('If', 'SCONJ'), ('I', 'PRON')],
 [('I', 'PRON'), ('could', 'AUX')],
 [('could', 'AUX'), ('lay', 'VERB')],
 [('lay', 'VERB'), ('my', 'PRON')],
 [('my', 'PRON'), ('hands', 'NOUN')],
 [('hands', 'NOUN'), ('on', 'ADP')],
 [('on', 'ADP'), ('those', 'DET')],
 [('those', 'DET'), ('letters', 'NOUN')],
 [('letters', 'NOUN'), ('I', 'PRON')],
 [('I', 'PRON'), ('could', 'AUX')],
 [('could', 'AUX'), ('make', 'VERB')],
 [('make', 'VERB'), ('a', 'DET')],
 [('a', 'DET'), ('fortune', 'NOUN')],
 [('fortune', 'NOUN'), ('Of', 'ADV')],
 [('Of', 'ADV'), ('course', 'ADV')],
 [('course', 'ADV'), ('not', 'PART')],
 [('not', 'PART'), ('What', 'PRON')],
 [('What', 'PRON'), ('upsets', 'VERB')],
 [('upsets', 'VERB'), ('me', 'PRON')],
 [('me', 'PRON'), ('is', 'AUX')],
 [('is', 'AUX'), ('the', 'DET')],
 [('the', 'DET'), ('fact', 'NOUN')],
 [('fact', 'NOUN'), ('that', 'SCONJ')],
 [('that', 'SCONJ'), ('Ugarte', 'PROPN')],
 [('Ugarte', 'PROPN'), ('is', 'AUX')],
 [('is', 'AUX'), ('dead', 'ADJ')],
 [('dead', 'ADJ'), ('and', 'CCONJ')],
 [('and', 'CCONJ'), ('no', 'DET')],
 [('no', 'DET'), ('one', 'NOUN')],
 [('one', 'NOUN'), ('knows', 'VERB')],
 [('knows', 'VERB'), ('where', 'SCONJ')],
 [('where', 'SCONJ'), ('those', 'DET')],
 [('those', 'DET'), ('letters', 'NOUN')],
 [('letters', 'NOUN'), ('of', 'ADP')],
 [('of', 'ADP'), ('transit', 'NOUN')],
 [('transit', 'NOUN'), ('are', 'AUX')],
 [('are', 'AUX'), ('The', 'DET')],
 [('The', 'DET'), ('bourbon', 'NOUN')],
 [('bourbon', 'NOUN'), ('The', 'DET')],
 [('The', 'DET'), ('news', 'NOUN')],
 [('news', 'NOUN'), ('about', 'ADP')],
 [('about', 'ADP'), ('Ugarte', 'PROPN')],
 [('Ugarte', 'PROPN'), ('upset', 'VERB')],
 [('upset', 'VERB'), ('me', 'PRON')],
 [('me', 'PRON'), ('very', 'ADV')],
 [('very', 'ADV'), ('much', 'ADV')],
 [('much', 'ADV'), ('Carrying', 'VERB')],
 [('Carrying', 'VERB'), ('charges', 'NOUN')],
 [('charges', 'NOUN'), ('my', 'PRON')],
 [('my', 'PRON'), ('boy', 'NOUN')],
 [('boy', 'NOUN'), ('carrying', 'VERB')],
 [('carrying', 'VERB'), ('charges', 'NOUN')],
 [('charges', 'NOUN'), ('Here', 'ADV')],
 [('Here', 'ADV'), ('sit', 'VERB')],
 [('sit', 'VERB'), ('down', 'ADP')],
 [('down', 'ADP'), ('There', 'PRON')],
 [('There', 'PRON'), ('something', 'PRON')],
 [('something', 'PRON'), ('I', 'PRON')],
 [('I', 'PRON'), ('want', 'VERB')],
 [('want', 'VERB'), ('to', 'PART')],
 [('to', 'PART'), ('talk', 'VERB')],
 [('talk', 'VERB'), ('over', 'ADP')],
 [('over', 'ADP'), ('with', 'ADP')],
 [('with', 'ADP'), ('you', 'PRON')],
 [('you', 'PRON'), ('anyhow', 'ADV')],
 [('anyhow', 'ADV'), ('No', 'DET')],
 [('No', 'DET'), ('hurry', 'NOUN')],
 [('hurry', 'NOUN'), ('I', 'PRON')],
 [('I', 'PRON'), ('have', 'VERB')],
 [('have', 'VERB'), ('it', 'PRON')],
 [('it', 'PRON'), ('sent', 'VERB')],
 [('sent', 'VERB'), ('over', 'ADP')],
 [('over', 'ADP'), ('Have', 'VERB')],
 [('Have', 'VERB'), ('a', 'DET')],
 [('a', 'DET'), ('drink', 'NOUN')],
 [('drink', 'NOUN'), ('with', 'ADP')],
 [('with', 'ADP'), ('me', 'PRON')],
 [('me', 'PRON'), ('My', 'PRON')],
 [('My', 'PRON'), ('dear', 'ADJ')],
 [('dear', 'ADJ'), ('Rick', 'PROPN')],
 [('Rick', 'PROPN'), ('when', 'SCONJ')],
 [('when', 'SCONJ'), ('will', 'AUX')],
 [('will', 'AUX'), ('you', 'PRON')],
 [('you', 'PRON'), ('realize', 'VERB')],
 [('realize', 'VERB'), ('that', 'SCONJ')],
 [('that', 'SCONJ'), ('in', 'ADP')],
 [('in', 'ADP'), ('this', 'DET')],
 [('this', 'DET'), ('world', 'NOUN')],
 [('world', 'NOUN'), ('today', 'NOUN')],
 [('today', 'NOUN'), ('isolationism', 'NOUN')],
 [('isolationism', 'NOUN'), ('is', 'AUX')],
 [('is', 'AUX'), ('no', 'ADV')],
 [('no', 'ADV'), ('longer', 'ADV')],
 [('longer', 'ADV'), ('a', 'DET')],
 [('a', 'DET'), ('practical', 'ADJ')],
 [('practical', 'ADJ'), ('policy', 'NOUN')],
 [('policy', 'NOUN'), ('Suppose', 'VERB')],
 [('Suppose', 'VERB'), ('we', 'PRON')],
 [('we', 'PRON'), ('ask', 'VERB')],
 [('ask', 'VERB'), ('Sam', 'PROPN')],
 [('Sam', 'PROPN'), ('Maybe', 'ADV')],
 [('Maybe', 'ADV'), ('he', 'PRON')],
 [('he', 'PRON'), ('like', 'VERB')],
 [('like', 'VERB'), ('to', 'PART')],
 [('to', 'PART'), ('make', 'VERB')],
 [('make', 'VERB'), ('a', 'DET')],
 [('a', 'DET'), ('change', 'NOUN')],
 [('change', 'NOUN'), ('That', 'PRON')],
 [('That', 'PRON'), ('too', 'ADV')],
 [('too', 'ADV'), ('bad', 'ADJ')],
 [('bad', 'ADJ'), ('That', 'PRON')],
 [('That', 'PRON'), ('Casablanca', 'PROPN')],
 [('Casablanca', 'PROPN'), ('leading', 'VERB')],
 [('leading', 'VERB'), ('commodity', 'NOUN')],
 [('commodity', 'NOUN'), ('In', 'ADP')],
 [('In', 'ADP'), ('refugees', 'NOUN')],
 [('refugees', 'NOUN'), ('alone', 'ADV')],
 [('alone', 'ADV'), ('we', 'PRON')],
 [('we', 'PRON'), ('could', 'AUX')],
 [('could', 'AUX'), ('make', 'VERB')],
 [('make', 'VERB'), ('a', 'DET')],
 [('a', 'DET'), ('fortune', 'NOUN')],
 [('fortune', 'NOUN'), ('if', 'SCONJ')],
 [('if', 'SCONJ'), ('you', 'PRON')],
 [('you', 'PRON'), ('would', 'AUX')],
 [('would', 'AUX'), ('work', 'VERB')],
 [('work', 'VERB'), ('with', 'ADP')],
 [('with', 'ADP'), ('me', 'PRON')],
 [('me', 'PRON'), ('through', 'ADP')],
 [('through', 'ADP'), ('the', 'DET')],
 [('the', 'DET'), ('black', 'ADJ')],
 [('black', 'ADJ'), ('market', 'NOUN')],
 [('market', 'NOUN'), ('What', 'PRON')],
 [('What', 'PRON'), ('do', 'AUX')],
 [('do', 'AUX'), ('you', 'PRON')],
 [('you', 'PRON'), ('want', 'VERB')],
 [('want', 'VERB'), ('for', 'ADP')],
 [('for', 'ADP'), ('Sam', 'PROPN')],
 [('Sam', 'PROPN'), ('You', 'PRON')],
 [('You', 'PRON'), ('have', 'AUX')],
 [('have', 'AUX'), ('heard', 'VERB')],
 [('heard', 'VERB'), ('my', 'PRON')],
 [('my', 'PRON'), ('offer', 'NOUN')],
 [('offer', 'NOUN'), ('Fine', 'INTJ')],
 [('Fine', 'INTJ'), ('but', 'CCONJ')],
 [('but', 'CCONJ'), ('I', 'PRON')],
 [('I', 'PRON'), ('would', 'AUX')],
 [('would', 'AUX'), ('like', 'VERB')],
 [('like', 'VERB'), ('to', 'PART')],
 [('to', 'PART'), ('buy', 'VERB')],
 [('buy', 'VERB'), ('your', 'PRON')],
 [('your', 'PRON'), ('cafe', 'NOUN')],
 [('cafe', 'NOUN'), ('Hello', 'PROPN')],
 [('Hello', 'PROPN'), ('Rick', 'PROPN')],
 [('Rick', 'PROPN'), ('It', 'PRON')],
 [('It', 'PRON'), ('was', 'AUX')],
 [('was', 'AUX'), ('gracious', 'ADJ')],
 [('gracious', 'ADJ'), ('of', 'ADP')],
 [('of', 'ADP'), ('you', 'PRON')],
 [('you', 'PRON'), ('to', 'PART')],
 [('to', 'PART'), ('share', 'VERB')],
 [('share', 'VERB'), ('it', 'PRON')],
 [('it', 'PRON'), ('with', 'ADP')],
 [('with', 'ADP'), ('me', 'PRON')],
 [('me', 'PRON'), ('Good', 'ADJ')],
 [('Good', 'ADJ'), ('day', 'NOUN')],
 [('day', 'NOUN'), ('Mademoiselle', 'PROPN')],
 [('Mademoiselle', 'PROPN'), ('Monsieur', 'PROPN')],
 [('Monsieur', 'PROPN'), ('He', 'PRON')],
 [('He', 'PRON'), ('is', 'AUX')],
 [('is', 'AUX'), ('a', 'DET')],
 [('a', 'DET'), ('difficult', 'ADJ')],
 [('difficult', 'ADJ'), ('customer', 'NOUN')],
 [('customer', 'NOUN'), ('that', 'SCONJ')],
 [('that', 'SCONJ'), ('Rick', 'PROPN')],
 [('Rick', 'PROPN'), ('One', 'NUM')],
 [('One', 'NUM'), ('never', 'ADV')],
 [('never', 'ADV'), ('knows', 'VERB')],
 [('knows', 'VERB'), ('what', 'PRON')],
 [('what', 'PRON'), ('he', 'PRON')],
 [('he', 'PRON'), ('do', 'VERB')],
 [('do', 'VERB'), ('or', 'CCONJ')],
 [('or', 'CCONJ'), ('why', 'SCONJ')],
 [('why', 'SCONJ'), ('But', 'CCONJ')],
 [('But', 'CCONJ'), ('it', 'PRON')],
 [('it', 'PRON'), ('is', 'AUX')],
 [('is', 'AUX'), ('worth', 'ADJ')],
 [('worth', 'ADJ'), ('a', 'DET')],
 [('a', 'DET'), ('chance', 'NOUN')],
 [('chance', 'NOUN'), ('Not', 'PART')],
 [('Not', 'PART'), ('for', 'ADP')],
 [('for', 'ADP'), ('sure', 'ADJ')],
 [('sure', 'ADJ'), ('Monsieur', 'PROPN')],
 [('Monsieur', 'PROPN'), ('but', 'CCONJ')],
 [('but', 'CCONJ'), ('I', 'PRON')],
 [('I', 'PRON'), ('will', 'AUX')],
 [('will', 'AUX'), ('venture', 'VERB')],
 [('venture', 'VERB'), ('to', 'PART')],
 [('to', 'PART'), ('guess', 'VERB')],
 [('guess', 'VERB'), ('that', 'SCONJ')],
 [('that', 'SCONJ'), ('Ugarte', 'PROPN')],
 [('Ugarte', 'PROPN'), ('left', 'VERB')],
 [('left', 'VERB'), ('those', 'DET')],
 [('those', 'DET'), ('letters', 'NOUN')],
 [('letters', 'NOUN'), ('with', 'ADP')],
 [('with', 'ADP'), ('Monsieur', 'PROPN')],
 [('Monsieur', 'PROPN'), ('Rick', 'PROPN')],
 [('Rick', 'PROPN'), ('Those', 'DET')],
 [('Those', 'DET'), ('letters', 'NOUN')],
 [('letters', 'NOUN'), ('were', 'AUX')],
 [('were', 'AUX'), ('not', 'PART')],
 [('not', 'PART'), ('found', 'VERB')],
 [('found', 'VERB'), ('on', 'ADP')],
 [('on', 'ADP'), ('Ugarte', 'PROPN')],
 [('Ugarte', 'PROPN'), ('when', 'SCONJ')],
 [('when', 'SCONJ'), ('they', 'PRON')],
 [('they', 'PRON'), ('arrested', 'VERB')],
 [('arrested', 'VERB'), ('him', 'PRON')],
 [('him', 'PRON'), ('I', 'PRON')],
 [('I', 'PRON'), ('observe', 'VERB')],
 [('observe', 'VERB'), ('that', 'SCONJ')],
 [('that', 'SCONJ'), ('you', 'PRON')],
 [('you', 'PRON'), ('in', 'ADP')],
 [('in', 'ADP'), ('one', 'NUM')],
 [('one', 'NUM'), ('respect', 'NOUN')],
 [('respect', 'NOUN'), ('are', 'AUX')],
 [('are', 'AUX'), ('a', 'DET')],
 [('a', 'DET'), ('very', 'ADV')],
 [('very', 'ADV'), ('fortunate', 'ADJ')],
 [('fortunate', 'ADJ'), ('man', 'NOUN')],
 [('man', 'NOUN'), ('Monsieur', 'PROPN')],
 [('Monsieur', 'PROPN'), ('I', 'PRON')],
 [('I', 'PRON'), ('am', 'AUX')],
 [('am', 'AUX'), ('moved', 'VERB')],
 [('moved', 'VERB'), ('to', 'PART')],
 [('to', 'PART'), ('make', 'VERB')],
 [('make', 'VERB'), ('one', 'NUM')],
 [('one', 'NUM'), ('more', 'ADJ')],
 [('more', 'ADJ'), ('suggestion', 'NOUN')],
 [('suggestion', 'NOUN'), ('why', 'SCONJ')],
 [('why', 'SCONJ'), ('I', 'PRON')],
 [('I', 'PRON'), ('do', 'AUX')],
 [('do', 'AUX'), ('not', 'PART')],
 [('not', 'PART'), ('know', 'VERB')],
 [('know', 'VERB'), ('because', 'SCONJ')],
 [('because', 'SCONJ'), ('it', 'PRON')],
 [('it', 'PRON'), ('can', 'AUX')],
 [('can', 'AUX'), ('not', 'PART')],
 [('not', 'PART'), ('possibly', 'ADV')],
 [('possibly', 'ADV'), ('profit', 'VERB')],
 [('profit', 'VERB'), ('me', 'PRON')],
 [('me', 'PRON'), ('but', 'CCONJ')],
 [('but', 'CCONJ'), ('have', 'AUX')],
 [('have', 'AUX'), ('you', 'PRON')],
 [('you', 'PRON'), ('heard', 'VERB')],
 [('heard', 'VERB'), ('about', 'ADP')],
 [('about', 'ADP'), ('Signor', 'PROPN')],
 [('Signor', 'PROPN'), ('Ugarte', 'PROPN')],
 [('Ugarte', 'PROPN'), ('and', 'CCONJ')],
 [('and', 'CCONJ'), ('the', 'DET')],
 [('the', 'DET'), ('letters', 'NOUN')],
 [('letters', 'NOUN'), ('of', 'ADP')],
 [('of', 'ADP'), ('transit', 'NOUN')],
 [('transit', 'NOUN'), ('Well', 'INTJ')],
 [('Well', 'INTJ'), ('good', 'ADJ')],
 [('good', 'ADJ'), ('luck', 'NOUN')],
 [('luck', 'NOUN'), ('But', 'CCONJ')],
 [('But', 'CCONJ'), ('be', 'AUX')],
 [('be', 'AUX'), ('careful', 'ADJ')],
 [('careful', 'ADJ'), ('You', 'PRON')],
 [('You', 'PRON'), ('know', 'VERB')],
 [('know', 'VERB'), ('you', 'PRON')],
 [('you', 'PRON'), ('being', 'AUX')],
 [('being', 'AUX'), ('shadowed', 'VERB')],
 [('shadowed', 'VERB'), ('We', 'PRON')],
 [('We', 'PRON'), ('might', 'AUX')],
 [('might', 'AUX'), ('as', 'ADV')],
 [('as', 'ADV'), ('well', 'ADV')],
 [('well', 'ADV'), ('be', 'AUX')],
 [('be', 'AUX'), ('frank', 'ADJ')],
 [('frank', 'ADJ'), ('Monsieur', 'PROPN')],
 [('Monsieur', 'PROPN'), ('It', 'PRON')],
 [('It', 'PRON'), ('will', 'AUX')],
 [('will', 'AUX'), ('take', 'VERB')],
 [('take', 'VERB'), ('a', 'DET')],
 [('a', 'DET'), ('miracle', 'NOUN')],
 [('miracle', 'NOUN'), ('to', 'PART')],
 [('to', 'PART'), ('get', 'VERB')],
 [('get', 'VERB'), ('you', 'PRON')],
 [('you', 'PRON'), ('out', 'ADP')],
 [('out', 'ADP'), ('of', 'ADP')],
 [('of', 'ADP'), ('Casablanca', 'PROPN')],
 [('Casablanca', 'PROPN'), ('And', 'CCONJ')],
 [('And', 'CCONJ'), ('the', 'DET')],
 [('the', 'DET'), ('Germans', 'PROPN')],
 [('Germans', 'PROPN'), ('have', 'AUX')],
 [('have', 'AUX'), ('outlawed', 'VERB')],
 [('outlawed', 'VERB'), ('miracles', 'NOUN')],
 [('miracles', 'NOUN'), ('As', 'ADP')],
 [('As', 'ADP'), ('leader', 'NOUN')],
 [('leader', 'NOUN'), ('of', 'ADP')],
 [('of', 'ADP'), ('all', 'DET')],
 [('all', 'DET'), ('illegal', 'ADJ')],
 [('illegal', 'ADJ'), ('activities', 'NOUN')],
 [('activities', 'NOUN'), ('in', 'ADP')],
 [('in', 'ADP'), ('Casablanca', 'PROPN')],
 [('Casablanca', 'PROPN'), ('I', 'PRON')],
 [('I', 'PRON'), ('am', 'AUX')],
 [('am', 'AUX'), ('an', 'DET')],
 [('an', 'DET'), ('influential', 'ADJ')],
 [('influential', 'ADJ'), ('and', 'CCONJ')],
 [('and', 'CCONJ'), ('respected', 'ADJ')],
 [('respected', 'ADJ'), ('man', 'NOUN')],
 [('man', 'NOUN'), ('It', 'PRON')],
 [('It', 'PRON'), ('would', 'AUX')],
 [('would', 'AUX'), ('not', 'PART')],
 [('not', 'PART'), ('be', 'AUX')],
 [('be', 'AUX'), ('worth', 'ADJ')],
 [('worth', 'ADJ'), ('my', 'PRON')],
 [('my', 'PRON'), ('life', 'NOUN')],
 [('life', 'NOUN'), ('to', 'PART')],
 [('to', 'PART'), ('do', 'VERB')],
 [('do', 'VERB'), ('anything', 'PRON')],
 [('anything', 'PRON'), ('for', 'ADP')],
 [('for', 'ADP'), ('Monsieur', 'PROPN')],
 [('Monsieur', 'PROPN'), ('Laszlo', 'PROPN')],
 [('Laszlo', 'PROPN'), ('You', 'PRON')],
 [('You', 'PRON'), ('however', 'ADV')],
 [('however', 'ADV'), ('are', 'AUX')],
 [('are', 'AUX'), ('a', 'DET')],
 [('a', 'DET'), ('different', 'ADJ')],
 [('different', 'ADJ'), ('matter', 'NOUN')]]

# Define a function to get the neighboring words based on bigrams
def get_neighbor_words(keyword, bigrams, pos_label = None):    
    neighbor_words = []
    keyword = keyword.lower()    
    for bigram in bigrams:       
        # extract just the lowercased words (not the labels) for each bigram
        words = [word.lower() for word, label in bigram]                
        # check to see if keyword is in the bigram
        if keyword in words:         
            for word, label in bigram:
                if word.lower() != keyword: # focusing on the neighbor word, not the keyword
                    neighbor_words.append(word.lower())    
    # return the word list after sorting it
    return Counter(neighbor_words).most_common()

# Get the neighboring words of the character Rick
get_neighbor_words("Rick", bigrams)

[('stay', 1),
 ('would', 1),
 ('be', 1),
 ('without', 1),
 ('enough', 1),
 ('do', 1),
 ('partner', 1),
 ('i', 1),
 ('dear', 1),
 ('when', 1),
 ('hello', 1),
 ('it', 1),
 ('that', 1),
 ('one', 1),
 ('monsieur', 1),
 ('those', 1)]

Task 4

# Print the neighboring words of the character Rick in each character's lines of the movie Casablanca and the characters' names.
# Write a loop through the movie_casablanca dataframe, copy the codes above of creating a list of tokens and POS labels 
# from each line if the token is a word, and then get all bigram of the lines. Finally, print movie_casablanca['cname'] of 
# that line, along with the neighboring words of Rick in that line.

for i in range(len(movie_casablanca['lines'])):
    tokens_and_labels = [(token.text, token.pos_) for token in nlp(movie_casablanca['lines'][i]) if token.is_alpha]
    bigrams = get_bigrams(tokens_and_labels)
    print(movie_casablanca['cname'][i], get_neighbor_words('Rick',bigrams))

ANNINA [('monsieur', 3), ('i', 1), ('what', 1)]
FERRARI [('stay', 1), ('would', 1), ('be', 1), ('without', 1), ('enough', 1), ('do', 1), ('partner', 1), ('i', 1), ('dear', 1), ('when', 1), ('hello', 1), ('it', 1), ('that', 1), ('one', 1), ('monsieur', 1), ('those', 1)]
ILSA [('no', 3), ('i', 3), ('is', 2), ('what', 2), ('oh', 1), ('but', 1), ('the', 1), ('will', 1), ('yes', 1), ('story', 1), ('do', 1), ('me', 1), ('it', 1), ('not', 1), ('some', 1), ('goodbye', 1), ('god', 1), ('about', 1), ('with', 1), ('he', 1), ('hello', 1), ('who', 1)]
LASZLO [('day', 1), ('do', 1), ('about', 1), ('in', 1)]
RENAULT [('and', 3), ('in', 2), ('with', 2), ('is', 2), ('about', 2), ('you', 2), ('there', 2), ('earlier', 1), ('met', 1), ('mademoiselle', 1), ('but', 1), ('sam', 1), ('monsieur', 1), ('victor', 1), ('if', 1), ('has', 1), ('of', 1), ('courage', 1), ('comes', 1), ('this', 1), ('at', 1), ('everybody', 1), ('to', 1), ('realizing', 1), ('well', 1), ('later', 1), ('huh', 1), ('germans', 1), ('have', 1), ('no', 1), ('never', 1), ('makes', 1), ('a', 1), ('half', 1), ('laszlo', 1), ('casablanca', 1), ('that', 1), ('know', 1), ('we', 1), ('less', 1)]
RICK [('owe', 1), ('a', 1)]
SAM []
STRASSER [('do', 1), ('i', 1), ('about', 1), ('himself', 1)]
UGARTE [('know', 2), ('hide', 1), ('me', 1), ('do', 1), ('something', 1), ('help', 1), ('i', 1), ('well', 1), ('after', 1), ('person', 1), ('if', 1), ('watching', 1), ('hello', 1)]

Task 5

What insights can be derived from the movie dialogue dataset using the Named Entity Recognition extraction method? Similarly, what insights can be gained through the Position-of-Speech tagging method? Please provide an example for each method, either based on the experiments covered above or by thinking of something else.

Some of the key takeaways we can pull from the movie dialogues using the Named entity recognition is being able to identify different types of entities in the movie dialogue. Although not 100% accurate this does serve to get an idea of how many People, Time, Cardinal, etc are in a given text. For instance if we are doing an analysis of the movie magnolia like in Task 1 but analyzing the motif of time we can analyze the occurrences of different words like today, tonight, years, etc. By doing this we can extract different themes from the text/movie. With position-of-speech tagging we can understand what common words are surrounding characters from a piece of work like a movie/text/etc. What we did in Task 4 is show different characters names with neighboring words. This can give a lot of context for the characters and their importance/role in the play such as Annina being referred to as 'Monsieur' multiple times showing that she is a woman of high importance maybe even high social status or class.