More web crawling capabilities

Registered by Dimitris Kalamaras on 2009-03-17

SocNetV should be able to crawl web pages (ie mailing list archives or facebook open graph) to simplify the work of network analyst/researcher.

Blueprint information

Status:
Started
Approver:
Dimitris Kalamaras
Priority:
Essential
Drafter:
Dimitris Kalamaras
Direction:
Approved
Assignee:
Dimitris Kalamaras
Definition:
Approved
Series goal:
Accepted for 2.x
Implementation:
Beta Available
Milestone target:
None
Started by
Dimitris Kalamaras on 2009-05-28

Related branches

Sprints

Whiteboard

Initial web crawler code has been added to SocNetV v. 0.70 and revamped in v1.6
The new code is good and pretty quick, but it could offer some other nice features :

1. Support for Mailing Lists emails crawling
2. Support for Facebook profile crawling

Ideas:
A. Can we add a menu item for the user to open the link in a browser? :)
B. Crawler should be able to count how many times a word appears in a page
Crawler code is there, but we need minor modifications to
a) form to add a textfield
b) code to count the used-defined

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.