p2pnet - rss feed: Meet SET p2p file sharing
p2pnet.net news:- Large p2p file transfers, including movies and music, could be much faster if p2p file sharing services are configured to share not only identical files, but also similar files.
That’s the conclusion of Dave Andersen, assistant professor of computer science at Carnegie Mellon, and Michael Kaminsky of Intel Research, Pittsburgh.
And they’ve done more than merely theorise. They’ve designed the Similarity-Enhanced Transfer (SET), a multi-source download system, to put their ideas into action.
“In tests based upon real files downloaded from today’s peer-to-peer networks, SET improved the transfer time of an MP3 music file by 71 percent,” they say. A larger 55-megabyte movie trailer went 30% faster using their techniques to draw from movie trailers that were 47% similar.
And, “This is a technique that I would like people to steal,” Andersen, a rock climber in his spare time, says.
The researchers, along with graduate student Himabindu Pucha of Purdue University, presented a paper describing SET and released the system code at the 4th Symposium on Networked Systems Design and Implementation in Cambridge, Massachusets, yesterday.
SET’s basic operation is similar to that of BitTorrent, says a Carnegie Mellon statement. Once the download of a data file is started, the source file is divided into smaller, unique chunks which are downloaded simultaneously from numerous sources that have the identical file. Then the chunks are reassembled into a single file.
“But while that process of downloading is under way, SET continues to search for similar files using a process called handprinting, inspired by techniques that have been used for clustering search results or detecting spam,” say the researchers. “A sampling technique is used to see if non-identical files contain chunks matching those of the desired file. Relevant chunks can then be downloaded from the similar files identified by this method.
Though Andersen and his colleagues stress they hope to implement SET in a service for sharing software or academic papers and have no intention of applying it themselves to movie- or music-sharing services, “it would make P2P transfers faster and more efficient,” he said.
Developers should, “just take the idea and use it in their own systems”.
No one knows the degree of similarity between data files stored in computers around the world, but analyses suggest the types of files most commonly shared are likely to contain a number of similar elements, says Carnegie Mellon, going on:
“Many music files, for instance, may differ only in the artist-and-title headers, but are otherwise 99 percent similar. Different versions of software packages likewise remain highly similar.”
Taking advantage of those similarities, “could speed downloads considerably”. For instance, if a US computer user wanted to download a German-language version of a popular movie, existing systems would probably download most of the movie from sources in Germany, says the researchers.
“But if the user could download from similar files, the user could retrieve most of the video from English versions readily available from U.S. sources, and download only the audio portion of the movie from the German sources,” say the researchers.
Click here for a .pdf of Exploiting Similarity for Multi-Source Downloads Using File Handprints.
Also See:
statement - Carnegie Mellon P2P System Could Speed Movie, Music Downloads; Technique Uses Similar Data Files, Not Just Identical Files, as Sources, April 10, 2007
If your Net access is blocked by government restrictions, try Psiphon from the Citizen Lab at thIs the endSurvey: How Did Copyright Infringement Become Equated with Robbery? (of the Net) nigh?zze University of Toronto’s Munk Centre for International Studies. Go here for the official download, here for the p2pnet download, and here for details. And if you’re Chinese and you’re looking for a way to access independent Internet news sources, try Freegate, the DIT program written to help Chinese citizens circumvent web site blocking outside of China. Download it here.
rss feed: http://p2pnet.net/p2p.rss | | Mobile - http://p2pnet.net/index-wml.php | | And use free p2pnet newsfeeds for your site
Tired of being treated like a criminal? They depend on you, not the other way around. Don’t buy their ‘product’. Do bug your local politicians. Use emails, snail-mail, phone calls, faxes, IM, stop them in the street, blog. And if you’re into organizing, organize petitions, organize demonstrations and then turn up on your local political rep’s doorstep, making sure you’ve contacted your local tv/radio station/newspaper in advance. Don’t just complain. Do something!
April 13th, 2007 at 12:22 am
I think he’s grossly overestimated the similarity of files in the real world. This is a pipe dream; this is “how I would do it, if only there really was similarity between files”.
April 13th, 2007 at 8:08 am
i disagree, it may be over estimated but i think it can acctually help. Think about searching for an mp3 file on a popular p2p network, you will often get meny results that are the same file size length and bitrate however due to differances in the id3 tags come up as different files.
The gain in practice is probably only going to be 10% or so but 10% is 10%…
April 13th, 2007 at 11:43 am
“Many music files, for instance, may differ only in the artist-and-title headers, but are otherwise 99 percent similar.”
I think we’ve known this for years now.