#17059 - 19/09/2000 01:02
Re: Automatic MP3 identification & how to build it
[Re: tfabris]
|
carpal tunnel
Registered: 19/05/1999
Posts: 3457
Loc: Palo Alto, CA
|
Mmmm, but this is how eTantrum want to make their money; quote from the website:
---- snip!
eTantrum's Dynamic Auditing System provides a powerful solution for real-time usage and demographic reporting. This technology generates customized reports on:
- Aggregate, anonymous consumer usage data
- Point-of-use behavioral analysis based on media usage coupled with; Demographic data (Sex, Age, and Zip code)
- This accurate and timely market research information is available in real-time and offers significant data that influences decisions made by the entertainment industry from pre-production through product release and distribution.
*eTantrum's technology is able to generate real-time demographic and usage information without compromising the user's name, email address, physical address, or any other personally identifiable information.
----
I don't see this as a problem personally, but I know people who would; however "non-identifiable" the source is, there's usually a problem a year down the line when someone finds out it's been logging ip addresses (not saying this will happen to eTantrum, but it seems to be pretty common in statistic gathering software!)
Hugo
|
Top
|
|
|
|
#17060 - 19/09/2000 08:14
Windows SongPrintDemo Binary
[Re: Dignan]
|
enthusiast
Registered: 09/11/1999
Posts: 398
Loc: Ashburn, VA
|
And please read all our troubles with identifying songs using PMA, since you say that it can and we have found that it can't.When you add songs into the PMA, it automatically starts identifying them in the background. If the song is not found, it will return "User-Defined" as the "File type". If the song is found in our database, it will return "eTantrum ID" for the file type. To get moreinfo, the song has to be found in our database. We have about 60,000 signatures in our database right now, and the number keeps growing daily. Attached is a freshly compiled SongSigDemo.exe along with the two DLL files you need to run it. Keep in mind this is just a simple test program just to demonstrate the technology. It doesn't have functionality to go back and modify the ID3 tags in a MP3 file. If you have any problems, questions, or requests, please feel free to msg here or privately. My email is robj@etantrum.com. --addition--- Apparently we can't attach files that big to posts. Here is a link to where you can grab it. http://www1.etantrum.com/~rjohnston/SongprintDemo-1.0.zip---- (O|||||O)
|
Top
|
|
|
|
#17061 - 19/09/2000 08:18
Re: Automatic MP3 identification & how to build it
[Re: altman]
|
enthusiast
Registered: 09/11/1999
Posts: 398
Loc: Ashburn, VA
|
I don't see this as a problem personally, but I know people who would; however "non-identifiable" the source is, there's usually a problem a year down the line when someone finds out it's been logging ip addresses (not saying this will happen to eTantrum, but it seems to be pretty common in statistic gathering software!)
I can definitely tell you that we do not log (or ever will log) IP addresses. I've seen the logs myself. :)
Buuuut then again, why should you believe me? :)
(O|||||O)
|
Top
|
|
|
|
#17062 - 19/09/2000 08:22
Re: Automatic MP3 identification & how to build it
[Re: tfabris]
|
enthusiast
Registered: 09/11/1999
Posts: 398
Loc: Ashburn, VA
|
I thought they did. Isn't that what the Personal Media Agent is? I don't know because I can't get the PMA to function from behind my firewall.
The PMA is far from done. :) We still have a pantload of stuff to implement/fix inside of it. It's been kinda put on the backburner since we've been pouring our entire effort into the Opensourced Songprint SDK recently.
I think that's only if you're using it in command-line mode under Linux or if you're integrating their code into your own. Isn't the Personal Media Agent supposed to handle that?
Nope. Well, not currently. The whole point of the technology is to *NOT* rely on ID3 tag data (which is inherently incorrect). Now, a feature where the PMA would go and REPLACE your ID3 tags with OUR tag data, that would be something that could be implemented, and we just hadn't thought of yet.
(O|||||O)
|
Top
|
|
|
|
#17063 - 19/09/2000 09:16
Re: Windows SongPrintDemo Binary
[Re: ClemsonJeep]
|
carpal tunnel
Registered: 08/03/2000
Posts: 12341
Loc: Sterling, VA
|
Thanks for clearing this up a little. Then how do you explain my "Across the Universe" thing? When the filename was "BEATLES -- Across the Universe.mp3", when it was loaded into the player it eventually filled out the track name field and the artist field. The others were blank. And there was no tag info on the file to start with. Then I renamed the file "song.mp3" just to test it. It didn't find it, and left the word "song" in the track name field. Then there's the test I ran with Beck's "Loser". In the very first post in this thread, the xml found the info for an arbitrary song off of Beck's Mellow Gold album, the same one that "Loser" is on. But when I loaded "Loser", it didn't find it. Instead it did the same as the Beatles tune, and put the filename, sans-extention, in the track name field. I'm still figuring out how this thing works. also, is it possible to return info to the database using PMA? How does it know if we don't want it to fill out our tags for us. DiGNAN
_________________________
Matt
|
Top
|
|
|
|
#17064 - 19/09/2000 11:02
Re: Windows SongPrintDemo Binary
[Re: Dignan]
|
enthusiast
Registered: 09/11/1999
Posts: 398
Loc: Ashburn, VA
|
Then how do you explain my "Across the Universe" thing? When the filename was "BEATLES -- Across the Universe.mp3", when it was loaded into the player it eventually filled out the track name field and the artist field. The others were blank. And there was no tag info on the file to start with.
Then I renamed the file "song.mp3" just to test it. It didn't find it, and left the word "song" in the track name field.
Then there's the test I ran with Beck's "Loser". In the very first post in this thread, the xml found the info for an arbitrary song off of Beck's Mellow Gold album, the same one that "Loser" is on. But when I loaded "Loser", it didn't find it. Instead it did the same as the Beatles tune, and put the filename, sans-extention, in the track name field.
I'm still figuring out how this thing works. also, is it possible to return info to the database using PMA? How does it know if we don't want it to fill out our tags for us.
Er.. When you say "tags" what exactly are you referring to? ID3 tags? What application are you referring to? The PMA or this test program? You need to be a bit more specific. :)
The PMA will do this in this order:
1) Parse the filename (Artist, Title, Album, etc) into the correct places 2) Parse the ID3 tag info (if any) 3) Attempt to identify the song with the song fingerprinting technology.
It will override any of the above "tags" with each of its predecessors.
In the PMA, if you do NOT have a "eTantrum ID" under the File status column, that song did NOT get identified with the songprint technology. It is relying on #2 or #1 from above.
The PMA does not have the option to enter song information if it is not found in our database (like the songprint SDK does). Once again, the PMA is older technology and the songprint SDK is newer.
My recommendation would be if you are just trying to see our identification system, use the program I linked to above. It will tell you if the songs you have are in our database, and if they aren't, you can add the information via a web page.
(O|||||O)
|
Top
|
|
|
|
#17065 - 19/09/2000 12:55
Re: Windows SongPrintDemo Binary
[Re: ClemsonJeep]
|
carpal tunnel
Registered: 08/03/2000
Posts: 12341
Loc: Sterling, VA
|
Okay, that's what I figured, despite the fact that on your website somewhere, it specifically says that PMA doesn't rely on the filename for identification. You should probably ammend that. And yes, I'm using the PMA program. And it still doesn't explain the Beck thing. This is a song that I'm guessing is in the database yet doesn't get recognized. But whatever. DiGNAN
_________________________
Matt
|
Top
|
|
|
|
#17066 - 19/09/2000 13:55
Re: Windows SongPrintDemo Binary
[Re: Dignan]
|
enthusiast
Registered: 09/11/1999
Posts: 398
Loc: Ashburn, VA
|
Okay, that's what I figured, despite the fact that on your website somewhere, it specifically says that PMA doesn't rely on the filename for identification. You should probably ammend that.
It doesn't rely on that. :) It will fall back on that if other identification methods fail, but in a perfect world, our database would contain every signature for every song ever created. Then we wouldn't need filenames or ID3 tags.
We're just in the process of building our database up right now, so we've got a.. er.. while.. to go. :)
And it still doesn't explain the Beck thing. This is a song that I'm guessing is in the database yet doesn't get recognized. But whatever.
Its possible. Each fingerprint is relatively unique. We perform some analysis on them ("alignment") which figures out what fingerprints are matched to what song data. It just might happen that you have an iteration that hasn't been aligned yet. If you use the SongPrintDemo-v1.0 program, it allows you to enter in the data to be submitted to our servers. Right now there is a lead time on when they get added, but we're fixing that right now so they are added automatically.
(O|||||O)
|
Top
|
|
|
|
#17067 - 19/09/2000 13:56
Re: Automatic MP3 identification
[Re: xml]
|
enthusiast
Registered: 09/11/1999
Posts: 398
Loc: Ashburn, VA
|
Anyone else tried this? Any successes, failures, comments?
Feel free to reply to this with your opinions!
(O|||||O)
|
Top
|
|
|
|
#17068 - 19/09/2000 17:50
Re: Windows SongPrintDemo Binary
[Re: ClemsonJeep]
|
stranger
Registered: 09/09/2000
Posts: 42
Loc: SC
|
Thanks for the Windows binary, ClemsonJeep. I actually would like to play with the source code, and I was glad to see that MS C++ project files were supplied in the source code zip file. My problem was that it required the glibc and fftw .lib files which had to be compiled as well and of course did not come with project files. Any chance you or anyone else could post those .lib files? Thanks again.
|
Top
|
|
|
|
#17069 - 20/09/2000 11:40
Re: Windows SongPrintDemo Binary
[Re: sc400]
|
enthusiast
Registered: 09/11/1999
Posts: 398
Loc: Ashburn, VA
|
Well, what is it you are trying to compile? Because you are going to need the header files for glib and fftw as well as the lib files. This is why we recommend you going to their site and downloading the source. If you can't do this for some reason let me know and I'll try to get the correct files together.
(O|||||O)
|
Top
|
|
|
|
#17070 - 20/09/2000 15:09
Re: Windows SongPrintDemo Binary
[Re: ClemsonJeep]
|
stranger
Registered: 09/09/2000
Posts: 42
Loc: SC
|
I did go to the site and download the source for glibc and fftw, and I now have the header files, but I can't compile the windows .lib files because there is no project file for them. I think that all I need now is the fftw.lib and glibc.lib files. Then I will be able toy with and compile the songprint demo.
|
Top
|
|
|
|
#17071 - 20/09/2000 22:00
Re: Windows SongPrintDemo Binary
[Re: ClemsonJeep]
|
carpal tunnel
Registered: 08/03/2000
Posts: 12341
Loc: Sterling, VA
|
Well, well, I tried out the songprint finally, and yes, it does work! fantastic! It did recognize the Beck song. It didn't recognize "Across the Universe" and "Would?" (by Alice In Chains). So I submitted the info to the database. However, because it recognized the Beck song, it leads me to believe that the PMA software just does not work in this aspect of the program. It just didn't recognize any of the songs at all. All you need is to make a simple, minimalist piece of software that finds the information and fills it out for you. Nothing more and nothing less. Yes, I would pay for this kind of software. And I tell you, I wouldn't pay for something like PMA. too cluttered, like WMP7 has become... But great job otherwise! I was very impressed once I finally saw my song ID'd by your system! I'm looking forward to anything you guys come out with in the future! DiGNAN
_________________________
Matt
|
Top
|
|
|
|
#17072 - 21/09/2000 14:44
Re: Windows SongPrintDemo Binary
[Re: Dignan]
|
stranger
Registered: 09/09/2000
Posts: 42
Loc: SC
|
I want the same thing, and I am willing to code it, but I need these .lib files. Or instructions on how to create them. And another thing that would be nice would be an easy, automated way to submit new signature-tag pairs, without having to do it interactively via the web. I (and many others) have tons of MP3s with perfectly good tags on them that I would love to contribute to your database, but I am not willing to spend the time to do it manually. If you are worried about bad tags overwriting good tags, perhaps you only allow automatic submition for tags that don't yet exist in the database at all. Edited by sc400 on 21/9/00 10:47 PM.
|
Top
|
|
|
|
#17073 - 22/09/2000 07:11
Re: Windows SongPrintDemo Binary
[Re: sc400]
|
enthusiast
Registered: 09/11/1999
Posts: 398
Loc: Ashburn, VA
|
I want the same thing, and I am willing to code it, but I need these .lib files. Or instructions on how to create them. And another thing that would be nice would be an easy, automated way to submit new signature-tag pairs, without having to do it interactively via the web. I (and many others) have tons of MP3s with perfectly good tags on them that I would love to contribute to your database, but I am not willing to spend the time to do it manually. If you are worried about bad tags overwriting good tags, perhaps you only allow automatic submition for tags that don't yet exist in the database at all.
We actually have a tool created such as this. It is called the gathertool and sofar it's used internally and been released to a few people to gather info about their song collections and send it to us. It basically uses the tag information from the MP3s or whatever as basic information and submits the data to our servers. We then do a process called "alignment" to make sure that the tag data is actually correct for that fingerprint before it is actually added to our production servers.
Last I checked the person working on the gathertool was working on a public version of it, but I'll check into that and contact you. If there isn't a public version out, I'm sure I could get a private copy to you if you want to do that. :)
(O|||||O)
|
Top
|
|
|
|
#17074 - 22/09/2000 20:42
Re: Windows SongPrintDemo Binary
[Re: ClemsonJeep]
|
carpal tunnel
Registered: 08/03/2000
Posts: 12341
Loc: Sterling, VA
|
heck, I have 3000 files in my collection. chances are you don't have info for at least 2000 of them. probably 2500. I've got pretty obscure music (so heck, maybe nobody else will ever access the stuff I provide ) but what the heck, can't hurt right? DiGNAN
_________________________
Matt
|
Top
|
|
|
|
#17075 - 26/09/2000 09:20
Re: Windows SongPrintDemo Binary
[Re: Dignan]
|
enthusiast
Registered: 05/09/2000
Posts: 210
Loc: Ipswich, MA
|
any news on this? I also have a ton of music that I would like to rename/reorganize. Since CDDB doesn't have year my search by year won't work unless I correct the tag.
Also track number would be f****ng great! I have several Live albums that got processed without track number and It's just not the same!
John
_________________________
___
John Turner
"It's easier to ask for forgiveness than to ask for permission"
|
Top
|
|
|
|
#17076 - 26/09/2000 09:47
Re: Windows SongPrintDemo Binary
[Re: jwtadmin]
|
carpal tunnel
Registered: 08/03/2000
Posts: 12341
Loc: Sterling, VA
|
Track number would be hard since there is no track number tag. DiGNAN
_________________________
Matt
|
Top
|
|
|
|
#17077 - 26/09/2000 09:51
Re: Windows SongPrintDemo Binary
[Re: Dignan]
|
carpal tunnel
Registered: 23/08/2000
Posts: 3826
Loc: SLC, UT, USA
|
Ermm...i definately could be wrong on this, but in MPTagger, there is a field for track number in the ID3v2 tags.... || loren.cox || 080000446
|
Top
|
|
|
|
#17078 - 26/09/2000 09:55
Re: Windows SongPrintDemo Binary
[Re: loren]
|
carpal tunnel
Registered: 20/12/1999
Posts: 31600
Loc: Seattle, WA
|
There is track number availabe in both V1 and V2 tags. In V2, it's a specific field. In V1, you can optionally use the last byte of the "Comment" field for the track number (in binary, not in ascii, that's why you sometimes see V1 comments with a little "block" character at the end). ___________ Tony Fabris
|
Top
|
|
|
|
#17079 - 26/09/2000 13:02
Re: Windows SongPrintDemo Binary
[Re: Dignan]
|
enthusiast
Registered: 09/11/1999
Posts: 398
Loc: Ashburn, VA
|
Track number would be hard since there is no track number tag.
It would also be difficult to maintain this kind of information because any given song can be on multiple albums, all with different track numbers. That is to say, the same exact song with the same fingerprint can occur on multiple albums with different track numbers. So it is a relative impossibility to keep track of this kind of information.
As an UPDATE:
I'm working on getting the GatherTool ready for public release. It will contain Win32 console and Win32 GUI interfaces, along with a Linux console interface. AFAIK it will be closed-source right now, but be open later on when we can clean up the code a bit. It's only just a frontend to the Songprint library anyway.
Ya'll will be the first to know about it when I am through with my work. :)
(O|||||O)
|
Top
|
|
|
|
#17080 - 26/09/2000 14:49
Re: Windows SongPrintDemo Binary
[Re: ClemsonJeep]
|
carpal tunnel
Registered: 08/03/2000
Posts: 12341
Loc: Sterling, VA
|
Awesome! You know, I think I'd like PMA better too if it didn't look so cluttered. Heck, I think it would work great if it were as streamlined as the songprint program you gave us! And I'd pay for it! (if it work) DiGNAN
_________________________
Matt
|
Top
|
|
|
|
#17081 - 26/09/2000 21:24
Re: Windows SongPrintDemo Binary
[Re: Dignan]
|
enthusiast
Registered: 09/11/1999
Posts: 398
Loc: Ashburn, VA
|
You know, I think I'd like PMA better too if it didn't look so cluttered. Heck, I think it would work great if it were as streamlined as the songprint program you gave us! And I'd pay for it! (if it work)
Yeah, I agree. Unfortunately I wasn't around for the PMA design, otherwise I would have started yelling about feature bloat and the fact that it is so very cluttered. I like a nice simple interface personally. I believe the PMA will be redesigned from ground up, and since I'll be on the design team, this will be one of our prime things to keep in mind.
The GatherTool is still in progress, BTW.. I'll keep you posted.
(O|||||O)
|
Top
|
|
|
|
#17082 - 26/09/2000 22:22
Re: Windows SongPrintDemo Binary
[Re: ClemsonJeep]
|
carpal tunnel
Registered: 08/03/2000
Posts: 12341
Loc: Sterling, VA
|
Cool beans. Way to go! DiGNAN
_________________________
Matt
|
Top
|
|
|
|
#17083 - 03/10/2000 15:45
Re: Windows SongPrintDemo Binary
[Re: ClemsonJeep]
|
stranger
Registered: 09/09/2000
Posts: 42
Loc: SC
|
Yes, track numbers will change since a song may appear on more than one album, but the album name also changes and we keep track of that. It is no different for track numbers. If a tag is representing album A, it will contain album A's album name and the track number the the song is at ON THAT ALBUM. Then you have to ask, well what is the correct album to use for a song that appears on multiple albums? The first one released, I guess. The space is there for a track number, to not attempt to include it (as well as the year) would be foolish. If someone decides they don't like the track number to be in there tags, I'm sure it could easily be batch removed with one of the many tag editors.
|
Top
|
|
|
|
#17084 - 19/10/2000 03:16
Re: Automatic MP3 identification
[Re: xml]
|
member
Registered: 30/08/2000
Posts: 157
Loc: London, UK
|
Great program... :)
Has any one managed to get it cross-compiled to run on the empeg itself...? I had a quick go and got a binary that ran but crashed when actually identifying a track... I won't post the error just yet as I will have another go at cross compiling it later once I have set up the environment a little better, but I was wondering if anyone else has had any more luck...
I was intending on running it over various badly tagged tracks on the empeg and having a script modify the *1 fids directly and then rebuilding the database... Can anyone see any other problems with this idea (assuming I can get the songprint cross-compiled)...?
Cheers
Kim
|
Top
|
|
|
|
#17085 - 20/10/2000 15:07
Re: Automatic MP3 identification
[Re: kimbotha]
|
enthusiast
Registered: 09/11/1999
Posts: 398
Loc: Ashburn, VA
|
Has any one managed to get it cross-compiled to run on the empeg itself...? I had a quick go and got a binary that ran but crashed when actually identifying a track... I won't post the error just yet as I will have another go at cross compiling it later once I have set up the environment a little better, but I was wondering if anyone else has had any more luck...
You won't be able to get it to run on the actual empeg box. The algorithm that is used to calculate the fingerprint for the song uses floating point mathematics and the ARM processor has no floating point unit. :)
(O|||||O)
ps: for those of you waiting on more about the library, our company is going through some interesting times. Downsizing to say the least. Work has been put on "pause" on this project to work on more important projects.
|
Top
|
|
|
|
#17086 - 21/10/2000 09:50
Re: Automatic MP3 identification
[Re: ClemsonJeep]
|
addict
Registered: 09/06/1999
Posts: 483
Loc: Guernsey
|
In reply to:
You won't be able to get it to run on the actual empeg box. The algorithm that is used to calculate the fingerprint for the song uses floating point mathematics and the ARM processor has no floating point unit. :)
It's true that the arm has no floating point unit, but the Linux kernel has a natty little FP emulation library. So you can run code containing FP, it's just rather slow... =)
Jazz (List 112, Mk2 12 gig #40. Mk1 4 gig #30. Mk3 1.6 16v)
_________________________
Jazz
(List 112, Mk2 42 gig #40. Mk1 4 gig #30. Mk3 1.6 16v)
|
Top
|
|
|
|
#17087 - 23/10/2000 03:03
Re: Automatic MP3 identification
[Re: Jazzwire]
|
member
Registered: 30/08/2000
Posts: 157
Loc: London, UK
|
Yes, I had thought this would be the problem... my next move (which I still haven't had the chance to try yet) is to compile it with libfloat (ftp://ftp.netwinder.org/users/s/scottb/libfloat/) and see if I can get it working with that... I don't really care how slow it is if it works... I can always leave it running overnight...
Do you think it will work with this...? I will give it a go as soon as I get a chance and work life butts out for a while...
Cheers
Kim
|
Top
|
|
|
|
#17088 - 03/12/2000 23:38
Re: Automatic MP3 identification
[Re: xml]
|
carpal tunnel
Registered: 08/03/2000
Posts: 12341
Loc: Sterling, VA
|
Just something I'd like to mention to you all. I emailed the guy who made Tag Studio to tell him about the Songprint stuff. He's going to look into it, but thought it was very cool and might add it to his program eventually. Pretty cool stuff. DiGNAN
_________________________
Matt
|
Top
|
|
|
|
|
|