CVV
CVV
CVV
All gists
Back to GitHub
Sign in
Sign up
Code
Revisions 1
log.txt
scrapy crawl toscrape-css
2016-12-09 18:57:32 [scrapy.utils.log] INFO: Scrapy 1.2.2 started (bot: quotesbot)
2016-12-09 18:57:32 [scrapy.utils.log] INFO: Overridden settings:
{'NEWSPIDER_MODULE': 'quotesbot.spiders', 'ROBOTSTXT_OBEY': True, 'BOT_NAME':
'quotesbot', 'SPIDER_MODULES': ['quotesbot.spiders']}
2016-12-09 18:57:32 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.telnet.TelnetConsole',
'scrapy.extensions.logstats.LogStats',
'scrapy.extensions.corestats.CoreStats']
2016-12-09 18:57:32 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.robotstxt.RobotsTxtMiddleware',
'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
'scrapy.downloadermiddlewares.retry.RetryMiddleware',
'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
'scrapy.downloadermiddlewares.stats.DownloaderStats']
2016-12-09 18:57:32 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
'scrapy.spidermiddlewares.referer.RefererMiddleware',
'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
'scrapy.spidermiddlewares.depth.DepthMiddleware']
2016-12-09 18:57:32 [scrapy.middleware] INFO: Enabled item pipelines:
[]
2016-12-09 18:57:32 [scrapy.core.engine] INFO: Spider opened
2016-12-09 18:57:32 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0
pages/min), scraped 0 items (at 0 items/min)
2016-12-09 18:57:32 [scrapy.extensions.telnet] DEBUG: Telnet console listening on
127.0.0.1:6023
2016-12-09 18:57:32 [scrapy.core.engine] DEBUG: Crawled (404) <GET
http://quotes.toscrape.com/robots.txt> (referer: None)
2016-12-09 18:57:32 [scrapy.core.engine] DEBUG: Crawled (200) <GET
http://quotes.toscrape.com/> (referer: None)
2016-12-09 18:57:32 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/>
{'text': '“The world as we have created it is a process of our thinking. It cannot
be changed without changing our thinking.”', 'tags': ['change', 'deep-thoughts',
'thinking', 'world'], 'author': 'Albert Einstein'}
2016-12-09 18:57:32 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/>
{'text': '“It is our choices, Harry, that show what we truly are, far more than our
abilities.”', 'tags': ['abilities', 'choices'], 'author': 'J.K. Rowling'}
2016-12-09 18:57:32 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/>
{'text': '“There are only two ways to live your life. One is as though nothing is a
miracle. The other is as though everything is a miracle.”', 'tags':
['inspirational', 'life', 'live', 'miracle', 'miracles'], 'author': 'Albert
Einstein'}
2016-12-09 18:57:32 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/>
{'text': '“The person, be it gentleman or lady, who has not pleasure in a good
novel, must be intolerably stupid.”', 'tags': ['aliteracy', 'books', 'classic',
'humor'], 'author': 'Jane Austen'}
2016-12-09 18:57:32 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/>
{'text': "“Imperfection is beauty, madness is genius and it's better to be
absolutely ridiculous than absolutely boring.”", 'tags': ['be-yourself',
'inspirational'], 'author': 'Marilyn Monroe'}
2016-12-09 18:57:32 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/>
{'text': '“Try not to become a man of success. Rather become a man of value.”',
'tags': ['adulthood', 'success', 'value'], 'author': 'Albert Einstein'}
2016-12-09 18:57:32 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/>
{'text': '“It is better to be hated for what you are than to be loved for what you
are not.”', 'tags': ['life', 'love'], 'author': 'André Gide'}
2016-12-09 18:57:32 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/>
{'text': "“I have not failed. I've just found 10,000 ways that won't work.”",
'tags': ['edison', 'failure', 'inspirational', 'paraphrased'], 'author': 'Thomas A.
Edison'}
2016-12-09 18:57:32 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/>
{'text': "“A woman is like a tea bag; you never know how strong it is until it's in
hot water.”", 'tags': ['misattributed-eleanor-roosevelt'], 'author': 'Eleanor
Roosevelt'}
2016-12-09 18:57:32 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/>
{'text': '“A day without sunshine is like, you know, night.”', 'tags': ['humor',
'obvious', 'simile'], 'author': 'Steve Martin'}
2016-12-09 18:57:33 [scrapy.core.engine] DEBUG: Crawled (200) <GET
http://quotes.toscrape.com/page/2/> (referer: http://quotes.toscrape.com/)
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/2/>
{'text': "“This life is what you make it. No matter what, you're going to mess up
sometimes, it's a universal truth. But the good part is you get to decide how
you're going to mess it up. Girls will be your friends - they'll act like it
anyway. But just remember, some come, some go. The ones that stay with you through
everything - they're your true best friends. Don't let go of them. Also remember,
sisters make the best friends in the world. As for lovers, well, they'll come and
go too. And baby, I hate to say it, most of them - actually pretty much all of them
are going to break your heart, but you can't give up because if you give up, you'll
never find your soulmate. You'll never find that half who makes you whole and that
goes for everything. Just because you fail once, doesn't mean you're gonna fail at
everything. Keep trying, hold on, and always, always, always believe in yourself,
because if you don't, then who will, sweetie? So keep your head high, keep your
chin up, and most importantly, keep smiling, because life's a beautiful thing and
there's so much to smile about.”", 'tags': ['friends', 'heartbreak',
'inspirational', 'life', 'love', 'sisters'], 'author': 'Marilyn Monroe'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/2/>
{'text': '“It takes a great deal of bravery to stand up to our enemies, but just as
much to stand up to our friends.”', 'tags': ['courage', 'friends'], 'author': 'J.K.
Rowling'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/2/>
{'text': "“If you can't explain it to a six year old, you don't understand it
yourself.”", 'tags': ['simplicity', 'understand'], 'author': 'Albert Einstein'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/2/>
{'text': "“You may not be her first, her last, or her only. She loved before she
may love again. But if she loves you now, what else matters? She's not perfect—you
aren't either, and the two of you may never be perfect together but if she can make
you laugh, cause you to think twice, and admit to being human and making mistakes,
hold onto her and give her the most you can. She may not be thinking about you
every second of the day, but she will give you a part of her that she knows you can
break—her heart. So don't hurt her, don't change her, don't analyze and don't
expect more than she can give. Smile when she makes you happy, let her know when
she makes you mad, and miss her when she's not there.”", 'tags': ['love'],
'author': 'Bob Marley'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/2/>
{'text': '“I like nonsense, it wakes up the brain cells. Fantasy is a necessary
ingredient in living.”', 'tags': ['fantasy'], 'author': 'Dr. Seuss'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/2/>
{'text': '“I may not have gone where I intended to go, but I think I have ended up
where I needed to be.”', 'tags': ['life', 'navigation'], 'author': 'Douglas Adams'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/2/>
{'text': "“The opposite of love is not hate, it's indifference. The opposite of art
is not ugliness, it's indifference. The opposite of faith is not heresy, it's
indifference. And the opposite of life is not death, it's indifference.”", 'tags':
['activism', 'apathy', 'hate', 'indifference', 'inspirational', 'love', 'opposite',
'philosophy'], 'author': 'Elie Wiesel'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/2/>
{'text': '“It is not a lack of love, but a lack of friendship that makes unhappy
marriages.”', 'tags': ['friendship', 'lack-of-friendship', 'lack-of-love', 'love',
'marriage', 'unhappy-marriage'], 'author': 'Friedrich Nietzsche'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/2/>
{'text': '“Good friends, good books, and a sleepy conscience: this is the ideal
life.”', 'tags': ['books', 'contentment', 'friends', 'friendship', 'life'],
'author': 'Mark Twain'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/2/>
{'text': '“Life is what happens to us while we are making other plans.”', 'tags':
['fate', 'life', 'misattributed-john-lennon', 'planning', 'plans'], 'author':
'Allen Saunders'}
2016-12-09 18:57:33 [scrapy.core.engine] DEBUG: Crawled (200) <GET
http://quotes.toscrape.com/page/3/> (referer: http://quotes.toscrape.com/page/2/)
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/3/>
{'text': '“I love you without knowing how, or when, or from where. I love you
simply, without problems or pride: I love you in this way because I do not know any
other way of loving but this, in which there is no I or you, so intimate that your
hand upon my chest is my hand, so intimate that when I fall asleep your eyes
close.”', 'tags': ['love', 'poetry'], 'author': 'Pablo Neruda'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/3/>
{'text': '“For every minute you are angry you lose sixty seconds of happiness.”',
'tags': ['happiness'], 'author': 'Ralph Waldo Emerson'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/3/>
{'text': '“If you judge people, you have no time to love them.”', 'tags':
['attributed-no-source'], 'author': 'Mother Teresa'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/3/>
{'text': '“Anyone who thinks sitting in church can make you a Christian must also
think that sitting in a garage can make you a car.”', 'tags': ['humor',
'religion'], 'author': 'Garrison Keillor'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/3/>
{'text': '“Beauty is in the eye of the beholder and it may be necessary from time
to time to give a stupid or misinformed beholder a black eye.”', 'tags': ['humor'],
'author': 'Jim Henson'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/3/>
{'text': '“Today you are You, that is truer than true. There is no one alive who is
Youer than You.”', 'tags': ['comedy', 'life', 'yourself'], 'author': 'Dr. Seuss'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/3/>
{'text': '“If you want your children to be intelligent, read them fairy tales. If
you want them to be more intelligent, read them more fairy tales.”', 'tags':
['children', 'fairy-tales'], 'author': 'Albert Einstein'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/3/>
{'text': '“It is impossible to live without failing at something, unless you live
so cautiously that you might as well not have lived at all - in which case, you
fail by default.”', 'tags': [], 'author': 'J.K. Rowling'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/3/>
{'text': '“Logic will get you from A to Z; imagination will get you everywhere.”',
'tags': ['imagination'], 'author': 'Albert Einstein'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/3/>
{'text': '“One good thing about music, when it hits you, you feel no pain.”',
'tags': ['music'], 'author': 'Bob Marley'}
2016-12-09 18:57:33 [scrapy.core.engine] DEBUG: Crawled (200) <GET
http://quotes.toscrape.com/page/4/> (referer: http://quotes.toscrape.com/page/3/)
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/4/>
{'text': "“The more that you read, the more things you will know. The more that you
learn, the more places you'll go.”", 'tags': ['learning', 'reading', 'seuss'],
'author': 'Dr. Seuss'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/4/>
{'text': '“Of course it is happening inside your head, Harry, but why on earth
should that mean that it is not real?”', 'tags': ['dumbledore'], 'author': 'J.K.
Rowling'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/4/>
{'text': '“The truth is, everyone is going to hurt you. You just got to find the
ones worth suffering for.”', 'tags': ['friendship'], 'author': 'Bob Marley'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/4/>
{'text': '“Not all of us can do great things. But we can do small things with great
love.”', 'tags': ['misattributed-to-mother-teresa', 'paraphrased'], 'author':
'Mother Teresa'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/4/>
{'text': '“To the well-organized mind, death is but the next great adventure.”',
'tags': ['death', 'inspirational'], 'author': 'J.K. Rowling'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/4/>
{'text': "“All you need is love. But a little chocolate now and then doesn't
hurt.”", 'tags': ['chocolate', 'food', 'humor'], 'author': 'Charles M. Schulz'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/4/>
{'text': "“We read to know we're not alone.”", 'tags': ['misattributed-to-c-s-
lewis', 'reading'], 'author': 'William Nicholson'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/4/>
{'text': '“Any fool can know. The point is to understand.”', 'tags': ['knowledge',
'learning', 'understanding', 'wisdom'], 'author': 'Albert Einstein'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/4/>
{'text': '“I have always imagined that Paradise will be a kind of library.”',
'tags': ['books', 'library'], 'author': 'Jorge Luis Borges'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/4/>
{'text': '“It is never too late to be what you might have been.”', 'tags':
['inspirational'], 'author': 'George Eliot'}
2016-12-09 18:57:33 [scrapy.core.engine] DEBUG: Crawled (200) <GET
http://quotes.toscrape.com/page/5/> (referer: http://quotes.toscrape.com/page/4/)
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/5/>
{'text': '“A reader lives a thousand lives before he dies, said Jojen. The man who
never reads lives only one.”', 'tags': ['read', 'readers', 'reading', 'reading-
books'], 'author': 'George R.R. Martin'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/5/>
{'text': '“You can never get a cup of tea large enough or a book long enough to
suit me.”', 'tags': ['books', 'inspirational', 'reading', 'tea'], 'author': 'C.S.
Lewis'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/5/>
{'text': '“You believe lies so you eventually learn to trust no one but
yourself.”', 'tags': [], 'author': 'Marilyn Monroe'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/5/>
{'text': '“If you can make a woman laugh, you can make her do anything.”', 'tags':
['girls', 'love'], 'author': 'Marilyn Monroe'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/5/>
{'text': '“Life is like riding a bicycle. To keep your balance, you must keep
moving.”', 'tags': ['life', 'simile'], 'author': 'Albert Einstein'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/5/>
{'text': '“The real lover is the man who can thrill you by kissing your forehead or
smiling into your eyes or just staring into space.”', 'tags': ['love'], 'author':
'Marilyn Monroe'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/5/>
{'text': "“A wise girl kisses but doesn't love, listens but doesn't believe, and
leaves before she is left.”", 'tags': ['attributed-no-source'], 'author': 'Marilyn
Monroe'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/5/>
{'text': '“Only in the darkness can you see the stars.”', 'tags': ['hope',
'inspirational'], 'author': 'Martin Luther King Jr.'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/5/>
{'text': '“It matters not what someone is born, but what they grow to be.”',
'tags': ['dumbledore'], 'author': 'J.K. Rowling'}
2016-12-09 18:57:33 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/5/>
{'text': '“Love does not begin and end the way we seem to think it does. Love is a
battle, love is a war; love is a growing up.”', 'tags': ['love'], 'author': 'James
Baldwin'}
2016-12-09 18:57:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET
http://quotes.toscrape.com/page/6/> (referer: http://quotes.toscrape.com/page/5/)
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/6/>
{'text': '“There is nothing I would not do for those who are really my friends. I
have no notion of loving people by halves, it is not my nature.”', 'tags':
['friendship', 'love'], 'author': 'Jane Austen'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/6/>
{'text': '“Do one thing every day that scares you.”', 'tags': ['attributed',
'fear', 'inspiration'], 'author': 'Eleanor Roosevelt'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/6/>
{'text': '“I am good, but not an angel. I do sin, but I am not the devil. I am just
a small girl in a big world trying to find someone to love.”', 'tags':
['attributed-no-source'], 'author': 'Marilyn Monroe'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/6/>
{'text': '“If I were not a physicist, I would probably be a musician. I often think
in music. I live my daydreams in music. I see my life in terms of music.”', 'tags':
['music'], 'author': 'Albert Einstein'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/6/>
{'text': '“If you only read the books that everyone else is reading, you can only
think what everyone else is thinking.”', 'tags': ['books', 'thought'], 'author':
'Haruki Murakami'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/6/>
{'text': '“The difference between genius and stupidity is: genius has its
limits.”', 'tags': ['misattributed-to-einstein'], 'author': 'Alexandre Dumas fils'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/6/>
{'text': "“He's like a drug for you, Bella.”", 'tags': ['drug', 'romance',
'simile'], 'author': 'Stephenie Meyer'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/6/>
{'text': '“There is no friend as loyal as a book.”', 'tags': ['books', 'friends',
'novelist-quotes'], 'author': 'Ernest Hemingway'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/6/>
{'text': '“When one door of happiness closes, another opens; but often we look so
long at the closed door that we do not see the one which has been opened for us.”',
'tags': ['inspirational'], 'author': 'Helen Keller'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/6/>
{'text': "“Life isn't about finding yourself. Life is about creating yourself.”",
'tags': ['inspirational', 'life', 'yourself'], 'author': 'George Bernard Shaw'}
2016-12-09 18:57:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET
http://quotes.toscrape.com/page/7/> (referer: http://quotes.toscrape.com/page/6/)
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/7/>
{'text': "“That's the problem with drinking, I thought, as I poured myself a drink.
If something bad happens you drink in an attempt to forget; if something good
happens you drink in order to celebrate; and if nothing happens you drink to make
something happen.”", 'tags': ['alcohol'], 'author': 'Charles Bukowski'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/7/>
{'text': '“You don’t forget the face of the person who was your last hope.”',
'tags': ['the-hunger-games'], 'author': 'Suzanne Collins'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/7/>
{'text': "“Remember, we're madly in love, so it's all right to kiss me anytime you
feel like it.”", 'tags': ['humor'], 'author': 'Suzanne Collins'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/7/>
{'text': '“To love at all is to be vulnerable. Love anything and your heart will be
wrung and possibly broken. If you want to make sure of keeping it intact you must
give it to no one, not even an animal. Wrap it carefully round with hobbies and
little luxuries; avoid all entanglements. Lock it up safe in the casket or coffin
of your selfishness. But in that casket, safe, dark, motionless, airless, it will
change. It will not be broken; it will become unbreakable, impenetrable,
irredeemable. To love is to be vulnerable.”', 'tags': ['love'], 'author': 'C.S.
Lewis'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/7/>
{'text': '“Not all those who wander are lost.”', 'tags': ['bilbo', 'journey',
'lost', 'quest', 'travel', 'wander'], 'author': 'J.R.R. Tolkien'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/7/>
{'text': '“Do not pity the dead, Harry. Pity the living, and, above all those who
live without love.”', 'tags': ['live-death-love'], 'author': 'J.K. Rowling'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/7/>
{'text': '“There is nothing to writing. All you do is sit down at a typewriter and
bleed.”', 'tags': ['good', 'writing'], 'author': 'Ernest Hemingway'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/7/>
{'text': '“Finish each day and be done with it. You have done what you could. Some
blunders and absurdities no doubt crept in; forget them as soon as you can.
Tomorrow is a new day. You shall begin it serenely and with too high a spirit to be
encumbered with your old nonsense.”', 'tags': ['life', 'regrets'], 'author': 'Ralph
Waldo Emerson'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/7/>
{'text': '“I have never let my schooling interfere with my education.”', 'tags':
['education'], 'author': 'Mark Twain'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/7/>
{'text': "“I have heard there are troubles of more than one kind. Some come from
ahead and some come from behind. But I've bought a big bat. I'm all ready you see.
Now my troubles are going to have troubles with me!”", 'tags': ['troubles'],
'author': 'Dr. Seuss'}
2016-12-09 18:57:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET
http://quotes.toscrape.com/page/8/> (referer: http://quotes.toscrape.com/page/7/)
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/8/>
{'text': '“If I had a flower for every time I thought of you...I could walk through
my garden forever.”', 'tags': ['friendship', 'love'], 'author': 'Alfred Tennyson'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/8/>
{'text': '“Some people never go crazy. What truly horrible lives they must lead.”',
'tags': ['humor'], 'author': 'Charles Bukowski'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/8/>
{'text': '“The trouble with having an open mind, of course, is that people will
insist on coming along and trying to put things in it.”', 'tags': ['humor', 'open-
mind', 'thinking'], 'author': 'Terry Pratchett'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/8/>
{'text': '“Think left and think right and think low and think high. Oh, the thinks
you can think up if only you try!”', 'tags': ['humor', 'philosophy'], 'author':
'Dr. Seuss'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/8/>
{'text': "“What really knocks me out is a book that, when you're all done reading
it, you wish the author that wrote it was a terrific friend of yours and you could
call him up on the phone whenever you felt like it. That doesn't happen much,
though.”", 'tags': ['authors', 'books', 'literature', 'reading', 'writing'],
'author': 'J.D. Salinger'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/8/>
{'text': '“The reason I talk to myself is because I’m the only one whose answers I
accept.”', 'tags': ['humor', 'insanity', 'lies', 'lying', 'self-indulgence',
'truth'], 'author': 'George Carlin'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/8/>
{'text': "“You may say I'm a dreamer, but I'm not the only one. I hope someday
you'll join us. And the world will live as one.”", 'tags': ['beatles',
'connection', 'dreamers', 'dreaming', 'dreams', 'hope', 'inspirational', 'peace'],
'author': 'John Lennon'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/8/>
{'text': '“I am free of all prejudice. I hate everyone equally. ”', 'tags':
['humor', 'sinister'], 'author': 'W.C. Fields'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/8/>
{'text': "“The question isn't who is going to let me; it's who is going to stop
me.”", 'tags': [], 'author': 'Ayn Rand'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/8/>
{'text': "“′Classic′ - a book which people praise and don't read.”", 'tags':
['books', 'classic', 'reading'], 'author': 'Mark Twain'}
2016-12-09 18:57:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET
http://quotes.toscrape.com/page/9/> (referer: http://quotes.toscrape.com/page/8/)
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/9/>
{'text': '“Anyone who has never made a mistake has never tried anything new.”',
'tags': ['mistakes'], 'author': 'Albert Einstein'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/9/>
{'text': "“A lady's imagination is very rapid; it jumps from admiration to love,
from love to matrimony in a moment.”", 'tags': ['humor', 'love', 'romantic',
'women'], 'author': 'Jane Austen'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/9/>
{'text': '“Remember, if the time should come when you have to make a choice between
what is right and what is easy, remember what happened to a boy who was good, and
kind, and brave, because he strayed across the path of Lord Voldemort. Remember
Cedric Diggory.”', 'tags': ['integrity'], 'author': 'J.K. Rowling'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/9/>
{'text': '“I declare after all there is no enjoyment like reading! How much sooner
one tires of any thing than of a book! -- When I have a house of my own, I shall be
miserable if I have not an excellent library.”', 'tags': ['books', 'library',
'reading'], 'author': 'Jane Austen'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/9/>
{'text': '“There are few people whom I really love, and still fewer of whom I think
well. The more I see of the world, the more am I dissatisfied with it; and every
day confirms my belief of the inconsistency of all human characters, and of the
little dependence that can be placed on the appearance of merit or sense.”',
'tags': ['elizabeth-bennet', 'jane-austen'], 'author': 'Jane Austen'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/9/>
{'text': '“Some day you will be old enough to start reading fairy tales again.”',
'tags': ['age', 'fairytales', 'growing-up'], 'author': 'C.S. Lewis'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/9/>
{'text': '“We are not necessarily doubting that God will do the best for us; we are
wondering how painful the best will turn out to be.”', 'tags': ['god'], 'author':
'C.S. Lewis'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/9/>
{'text': '“The fear of death follows from the fear of life. A man who lives fully
is prepared to die at any time.”', 'tags': ['death', 'life'], 'author': 'Mark
Twain'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/9/>
{'text': '“A lie can travel half way around the world while the truth is putting on
its shoes.”', 'tags': ['misattributed-mark-twain', 'truth'], 'author': 'Mark
Twain'}
2016-12-09 18:57:34 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/9/>
{'text': '“I believe in Christianity as I believe that the sun has risen: not only
because I see it, but because by it I see everything else.”', 'tags':
['christianity', 'faith', 'religion', 'sun'], 'author': 'C.S. Lewis'}
2016-12-09 18:57:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET
http://quotes.toscrape.com/page/10/> (referer: http://quotes.toscrape.com/page/9/)
2016-12-09 18:57:35 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/10/>
{'text': '“The truth." Dumbledore sighed. "It is a beautiful and terrible thing,
and should therefore be treated with great caution.”', 'tags': ['truth'], 'author':
'J.K. Rowling'}
2016-12-09 18:57:35 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/10/>
{'text': "“I'm the one that's got to die when it's time for me to die, so let me
live my life the way I want to.”", 'tags': ['death', 'life'], 'author': 'Jimi
Hendrix'}
2016-12-09 18:57:35 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/10/>
{'text': '“To die will be an awfully big adventure.”', 'tags': ['adventure',
'love'], 'author': 'J.M. Barrie'}
2016-12-09 18:57:35 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/10/>
{'text': '“It takes courage to grow up and become who you really are.”', 'tags':
['courage'], 'author': 'E.E. Cummings'}
2016-12-09 18:57:35 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/10/>
{'text': '“But better to get hurt by the truth than comforted with a lie.”',
'tags': ['life'], 'author': 'Khaled Hosseini'}
2016-12-09 18:57:35 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/10/>
{'text': '“You never really understand a person until you consider things from his
point of view... Until you climb inside of his skin and walk around in it.”',
'tags': ['better-life-empathy'], 'author': 'Harper Lee'}
2016-12-09 18:57:35 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/10/>
{'text': '“You have to write the book that wants to be written. And if the book
will be too difficult for grown-ups, then you write it for children.”', 'tags':
['books', 'children', 'difficult', 'grown-ups', 'write', 'writers', 'writing'],
'author': "Madeleine L'Engle"}
2016-12-09 18:57:35 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/10/>
{'text': '“Never tell the truth to people who are not worthy of it.”', 'tags':
['truth'], 'author': 'Mark Twain'}
2016-12-09 18:57:35 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/10/>
{'text': "“A person's a person, no matter how small.”", 'tags': ['inspirational'],
'author': 'Dr. Seuss'}
2016-12-09 18:57:35 [scrapy.core.scraper] DEBUG: Scraped from <200
http://quotes.toscrape.com/page/10/>
{'text': '“... a mind needs books as a sword needs a whetstone, if it is to keep
its edge.”', 'tags': ['books', 'mind'], 'author': 'George R.R. Martin'}
2016-12-09 18:57:35 [scrapy.core.engine] INFO: Closing spider (finished)
2016-12-09 18:57:35 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 2859,
'downloader/request_count': 11,
'downloader/request_method_count/GET': 11,
'downloader/response_bytes': 24819,
'downloader/response_count': 11,
'downloader/response_status_count/200': 10,
'downloader/response_status_count/404': 1,
'finish_reason': 'finished',
'finish_time': datetime.datetime(2016, 12, 9, 13, 57, 35, 23112),
'item_scraped_count': 100,
'log_count/DEBUG': 112,
'log_count/INFO': 7,
'request_depth_max': 9,
'response_received_count': 11,
'scheduler/dequeued': 10,
'scheduler/dequeued/memory': 10,
'scheduler/enqueued': 10,
'scheduler/enqueued/memory': 10,
'start_time': datetime.datetime(2016, 12, 9, 13, 57, 32, 469393)}
2016-12-09 18:57:35 [scrapy.core.engine] INFO: Spider closed (finished)
to join this conversation on GitHub. Already have an account? Sign in to comment
© 2020 GitHub, Inc.
Terms
Privacy
Security
Status
Help
Contact GitHub
Pricing
API
Training
Blog
About