Renaming files outside Slsk causes "Rescan Shares" to occasionally "lose" files

Submitted by monsieurX on Sat, 04/07/2012 - 05:32
Description: 
OS = Linux. (Important because I suspect this to be a Linux-only issue) Hi, whilst reorganizing some of my older stuff, this has become a real issue here. I noticed missing spaces, rogue characters and other things that did not belong in these filenames, so I renamed the files in my terminal app (e. g. konsole) and did a RESCAN SHARES. Then I added myself to the Userlist so I could see how my share will look to others. From 10 files originally in my folder X, after renaming a few of them SlskQt only showed 5 files in the folder, telling me about "5 files" being "deleted" from the folder. (True because since I renamed them, they're regarded as "deleted".) - Rescan shares, deleting browse tab, browsing myself again ... still 5 files. Not until I fully restarted SlskQt, my "lost" files automagically came back. Even unsharing->resharing sometimes (but not always) does not bring back said files. Currently, app restart is the only way out.
0
Your rating: None
0
No votes yet

Comments

Submitted by Nir on Sat, 04/07/2012 - 16:07.

I'm kinda inundated right now but I'll look into it as soon as I get the chance.

Thanks, Nir

Submitted by monsieurX on Sun, 04/08/2012 - 01:55.

Inundated? I hope that's not through me too, because I know I've been keeping spamming your forum for some time now. :) OTOH, times like Easter are days where there's really time to dive in-depth into some special matters, so that's where my "post overkill" originates from...

Addendum: it's even weirder than I thought: I made another experiment. I browsed myself, counted the files CHANGED NOTHING in said folder, and then I browsed myself again 2 hours later. You won't believe it: there were a handful of files less than before, as if something had "eaten" them up.

Submitted by Nir on Sun, 04/08/2012 - 01:54.

Hmm. That's very worrisome. I had something similar happen to me with an interim build from a couple of weeks ago, but I doubt that's the problem in your case. Here's what you can do to help me get started: Start the client, verify that all of your shared files appear to be there. Make a copy of the client configuration file, it's a file named soulseek-client.dat and it's usually in the application data folder. Search for the filename if you're not sure where it is. Once you find it, rename the copy to something distinct. Then, wait to see if you lose any files in your share again. If you do, repeat the process and name the file something else. If you manage to reproduce the problem and get those files, send them both to me via the Send File link on my user page. I'll be able to use them to see analyze the state of the data system at each point in the client's run.

And don't worry about flooding me! I appreciate your help and participation. It's not just you who's keeping me busy :) A lot of new issues have been brought up over the last two weeks, and there are things I've set out to do for a quite a while that I haven't gotten around to yet. It's just inevitable that certain things will take longer than they should.

Thanks, Nir

Submitted by monsieurX on Sun, 04/08/2012 - 05:45.

Well, if my eyes are still fully intact, it's *NOT* always the same files that "disappear"! That's another problem. So you can't just point your finger and say: that's the culprit(s). (Would of been too easy huh :p) So I will do as you propose and create some of these "diff" config files.

Also, I will try with both "new" and "old" client version. Please don't chop my head off now if I say that I mainly use the Jan 11 2012 build for reasons of higher stability. However, in this case I think it does not matter because I can't seem to have read anything about your fixing of any issues related to "file munching" in the meantime.

Submitted by Nir on Sun, 04/08/2012 - 02:19.

Actually, you don't have to go to the trouble. I'm seeing the bug over here. Crap. I'll figure it out and roll out a fix ASAP.

Thanks, Nir

Submitted by Nir on Sun, 04/08/2012 - 03:15.

Well, the good news is that it's a bug that triggers under fairly unusual circumstances. In the case of our particular problem, it has everything to do with browsing your own share, and should not occur otherwise except occasionally. The bad news is that it's a pervasive bug with the data system that can affect the client in numerous small and unexpected ways. Fixing this is absolutely my top priority right now. I will release an update as soon as I can.

Submitted by monsieurX on Sun, 04/08/2012 - 06:07.

>I will release an update as soon as I can.
You're just fantastic, thanks so much! You're giving a support beyond compare. I guess that's due to Linux people having given you motivation by eventually creeping out of their hideouts and showing you that your work was /not/ in vain at all and that your Linux client has actually quite a decent number of thankful users! (though silent, yes they *are* there, let alone those Mac guys)

Anyways, back to the issue:
>should not occur otherwise except occasionally.
Yes I think you're right this only occurs when you're browsing your own share. Maybe the majority of users would just think "who'd attempt to do a thing like that !?" However, I'd have my doubts anyone would want to have to browse shares from users that look like a virtual messie's home either! So in fact, this clever step has helped me eliminate lots of embarrassing things beforehand, simply by looking into that kind of virtual "How-the-customer-will-see-you mirror" in time :)

Submitted by Nir on Mon, 04/09/2012 - 01:22.

Hey monsieurX, hopefully you're still around. I put the entire data system through a pretty major overhaul. At this point I'm fairly certain the disappearing files issue should be resolved, but I'm more concerned with anything else that might go AWOL as a result of these changes. One way or another I have to release this within the next couple of days, but I'm curious to see if you might run into any new issues with this fix before then: SoulseekQt-data-loss-fix.tgz.

Thanks! Nir

Submitted by monsieurX on Mon, 04/09/2012 - 09:59.

Howdy Nir, sure thing I still exist! (lol) Thanks for the recent build, will try it immediately now. Well, while you're on it, there might be another bug at least still in my "uber-build" of Jan 11 2012. I guess it's because in "Browse" tab, I have my own share open as well as that of another user. Now something very strange happened (or frankly, I've been witnessing it for several days now): while browsing through the user's share (expand collapse and all that jazz), I notice that all of a sudden my mouse pointer gets unresponsive, as if under heavy CPU load. The solution: *MY* own share is being downloaded to myself! No you haven't misread anything: my username appears in "Downloads" with all files selected for download! (My download directory is different from my share directory, just FYI).

Seems to me like another side-effect in the "check own share by browsing yourself" category. Just for you to note: it has happened several times, not just once. And my tab was inactive, because I was browsing the user's share.

Submitted by Nir on Mon, 04/09/2012 - 13:59.

Can you try and see if this happens in the data loss fix build? Are you doing anything special in the share browse before this happens?

Submitted by monsieurX on Wed, 04/11/2012 - 06:14.

No I don't. Frankly, I had a suspicion that your data/file system overhaul could have already fixed it, but I thought I'd mentioned it anyway because this "data loss fix build" is ALPHA and others might even run into same problems using a client from the January--March period.

Currently, everything works. Thank you very much, your "alpha" build could even qualify as a RC.
Even another strange bug does no longer occur: "Download File" occasionally MADE slsk bail out and CRASH, whilst "Download Folder" always worked perfectly.

Nir, that was superb!

Submitted by Nir on Wed, 04/11/2012 - 16:11.

Thanks for the report monsieurX :) this makes me feel a lot better. I posted the data loss fix to the download page last night.

Cheers, Nir

Submitted by monsieurX on Tue, 05/01/2012 - 11:56.

Well, Nir, I think after weeks of extensive testing, I can say for sure that all your Linux builds before the data-loss fix build are pretty unusable!
They are because they do not show the real thing.

One day, I had to switch to the begin-of-January build because the "data loss" build suddenly would always segfault when starting up. (Funnily enough, after you have worked again with the stable build for awhile, quitting it, the "data loss" build will work again. The crash is perhaps caused by a corrupt configuration file, or one in partly illegal format)

Well, so I used said stable build again for the time being, and browsed a guy who had lots of 90s charts stuff.
The result was predictable.
At the beginning, it was possible to see the whole of files in one of the "years" folders (p_101.mp3 up to p_345.mp3) - there were no gaps. Then I used the filter on it (p_1) to get all the 1xx files. Soon, I got a reduced file set: p_101, p_105, p_113 and so on) . Lots of files missing in-between.

So the "data loss fix" build is perfect as long as it works. Once it gets into its segfault mood again, and you have to use the (so-called) "stable" client, you will realize that all the versions before the "data loss fix" build were pretty unusable (sad but true), because you simply got a reduced file set after a while.

I'm just posting this to let you know it is *NOT* only related to browsing yourself, but also to browsing other users you have in your list.

Submitted by Nir on Tue, 05/01/2012 - 18:47.

Hi monsieurX,

Once the crashes start happening, do they happen whenever you start the program? If so, I could really benefit from looking at your data file in that case. You can use the 'Send File' link on my user page to send it over my way. The file in question is .Soulseek in your home folder, though I suspect you know this already. Also know that the SoulseekQt executable is compiled with basic debug information, so if you start it in gdb you should be able to get a stack trace that shows where the crash occurs. If your client gets to a point where it crashes whenever you start it, I say send me the data file. If not, you can use gdb to get a stack trace and post it here. This should give me an idea of what's going wrong.

Regarding the loss of information in older versions, yes, the bug you helped me uncover can affect the data system in a number of unpredictable ways, and since it's used by the client across the board there's no telling how it might manifest. That's why it's important to always use the latest version. Hopefully we can keep hammering all the bugs out build by build until we end up with one you can use without any problems.

Thanks, Nir