crawley-project.github.io/index.html at master · crawley-project/crawley-project.github.io · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
---
layout: page
title: Crawley Project
---
<a href="https://github.com/jmg/crawley"><img style="position: absolute; top: 0; left: 992px; border: 0;"
                                              src="https://camo.githubusercontent.com/a6677b08c955af8400f44c6298f40e7d19cc5b2d/68747470733a2f2f73332e616d617a6f6e6177732e636f6d2f6769746875622f726962626f6e732f666f726b6d655f72696768745f677261795f3664366436642e706e67"
                                              alt="Fork me on GitHub"
                                              data-canonical-src="https://s3.amazonaws.com/github/ribbons/forkme_right_gray_6d6d6d.png"></a>

<div id="middle">

    <div id="content" class="content">


        <h2>Welcome to the crawley project web site!</h2>
        <br/>
        <iframe src="https://ghbtns.com/github-btn.html?user=jmg&repo=crawley&type=star&count=true&size=large"
                frameborder="0" scrolling="0" width="160px" height="30px"></iframe>
        <iframe src="https://ghbtns.com/github-btn.html?user=jmg&repo=crawley&type=watch&count=true&size=large&v=2"
                frameborder="0" scrolling="0" width="160px" height="30px"></iframe>


        <h3>Crawley is Pythonic Crawling / Scraping framework intented to change the way you think about extracting data
            from the internet.</h3>

        <div>
            <br/>
            <b>
                <div>Features:</div>
            </b>
            <br/>
            <ul>
                <li>High Speed WebCrawler built on Eventlet.</li>
                <li>Store you data in relational databases like Postgres, Mysql, Oracle, Sqlite.</li>
                <li>Export your data into Json, XML formats. <span class="label success">New</span></li>
                <li>Supports NoSQL databased like Mongodb and Couchdb. <span class="label success">New</span></li>
                <li>Command line tools.</li>
                <li>Extract data using your favourite tool. XPath or Pyquery (A Jquery-like library for python).</li>
                <li>Cookie Handlers for scraping login protected pages.</li>
                <li>Very easy to use (see the <a href="https://github.com/jmg/crawley/tree/0.2.x-stable/examples">examples</a>).
                </li>
            </ul>
            <div>We are proud to announce the <a
                    href="http://pypi.python.org/pypi?:action=display&name=crawley&version=0.2.3">release 0.2.3 of the
                crawley framework</a>.
            </div>
            <br/>

            <div>Satisfy Dependencies. In ubuntu just do:</div>
            <br/>
            <pre class="prettyprint">~$ apt-get install python-dev libxml2 libxslt1-dev</pre>

            <div>Then you can download the latest version from pip:</div>
            <br/>
            <pre class="prettyprint">~$ pip install crawley</pre>

            <div>Or clone the repository at github:</div>

            <br/>
            <pre class="prettyprint">~$ git clone git://github.com/jmg/crawley.git</pre>

            <h4>Where to start?</h4>
            <br/>

            <div>You can checkout first the <a href="http://packages.python.org/crawley/">documentation</a> and learn
                crawley with examples!. Also you can join the <a href="http://groups.google.com/group/crawley-users">crawley-users
                    group</a>. The community is in there to help you.
            </div>
            <br/>


            <h4>Do you wanna contribute?</h4>
            <br/>

            <div>Join the <a href="http://groups.google.com/group/crawley-developers">crawley-developers group</a>.
            </div>
            <br/>

            <div>Or make a financial donation via paypal.</div>
            <br/>

            <form action="https://www.paypal.com/cgi-bin/webscr" method="post"><input type="hidden" name="cmd"
                                                                                      value="_donations"><input
                    type="hidden" name="business" value="jmg.utn@gmail.com"><input type="hidden" name="lc"
                                                                                   value="US"><input type="hidden"
                                                                                                     name="item_name"
                                                                                                     value="crawley-project"><input
                    type="hidden" name="no_note" value="0"><input type="hidden" name="currency_code" value="USD"><input
                    type="hidden" name="bn" value="PP-DonationsBF:btn_donateCC_LG.gif:NonHostedGuest"><input
                    type="image" src="https://www.paypalobjects.com/en_US/i/btn/btn_donateCC_LG.gif" border="0"
                    name="submit" alt="PayPal - The safer, easier way to pay online!"><img alt="" border="0"
                                                                                           src="https://www.paypalobjects.com/es_XC/i/scr/pixel.gif"
                                                                                           width="1" height="1"></form>
        </div>

    </div>
    <br/>
    {% include recent_posts.html %}

</div>