Deleting and purging an index on a Synology DiskStation NAS

I was trying to set up my Synology DiskStation (DSM 6.2.3 as of writing) as a media player. I was happy to discover that there is a built in application for the DiskStation you can load up from the Package Centre called “Audio Station”, with matching apps on the Android and iPhone. I installed it, created a new account with read-only file access to my music files in a particular directory, and only application access to the Audio Station applicaiton.

All looked great until the Indexer started trawling the entire disk volume. This means that all sorts of things that were just audio files were now listed as music tracks, and if you have the same track in multiple formats, suddenly you have 2 copies of the same track in the app. As you can imagine, this makes playlists and shuffles really annoying to say the least.

So I went in to the “Indexing Service” on the control panel, and after trying multiple combinations of renaming, changing settings, deleting directories from the index and selecting where I wanted indexed, it became clear that the NAS doesn’t like removing things from the index, at least not with the amount of time I was willing to wait. I will also note at this point that searching for help on removing references from the index is problematic at best- there are a few articles on the index retaining deleted files or moved files at both locations.

At this point I left the indexer settings at my desired result – one specific directory tree on one volume for audio files only.

Next step? Dropping the old database seemed the way to go. This wasn’t easy, and I’m writing this to remind myself of what I had to do, in case I need to do similar for the file or photo applications, or simply need to reset the index again.

Note that each version of the DSM is different in how to achieve the outcome, so these are the next steps I followed.

First you need to stop applications using the index database. I stopped the “Audio Station” application by going into the “Package Centre” and selecting the application, then chosing “Stop” from the dropdown.

Next challenge was stopping the Indexe, which I did from a secure shell promptr:

super@DSM:/$ sudo synoservice -stop synoindexd
some service depend on service [synoindexd] is still running, use --hard-stop to stop all releated service
super@DSM:/$ sudo synoservice --hard-stop synoindexd
warn: The runkey of service [synoindexd] and its dependency services have been set to "No"

Then I needed to drop the database. The synoindex tool has a lot of features, but they seem to resemble what’s in the web GUI except there’s no help listing which directories have been indexed, so you can’t work out what to delete.

ssh super@DSM
super@DSM's password:
Could not chdir to home directory /var/services/homes/super: No such file or directory
super@DSM:/$ synoindex
Usage:
synoindex [OPTIONS]
Index Options:
-h, --help
this help text
-A dirpath
add a dir
-a filepath
add a file
-D dirpath
delete a dir
-d filepath
delete a file
-N new_dirpath old_dirpath
rename a dir
-n new_filepath old_filepath
rename a file
-R [all|media|photo|music|video|thumb|dirpath]
all: reindex all dirpath that registered in each package
media: reindex dirpath that registered in MediaIndex package
photo: reindex photo dirpath
music: reindex music dirpath
video: reindex video dirpath
thumb: check converted video of each video file
dirpath: reindex this specific dirpath
-R user:{user_name}
reindex personal photo dirpath
-R share:{share_name}
reindex share dirpath
-R [type_music|type_video|type_photo]
reindex dirpath that registered with specific type in MediaIndex
Package Index Options:
-P [MediaIndex|{package_name}] {index_option}
index operation only apply on this package
-p [MediaIndex|{package_name}] {index_option}
index operation apply all packages except for this package
File Index Options:
-f {index_option}
index operation apply on file index
-U photo
update photo images

So I tried to login as the database as the database user: postgres. Unfortunately I didn’t have a password for that account, and I couldn’t su postgres without the password. So I did the following workaround:

super@DSM:/$ sudo /bin/bash
Password:
bash-4.3# su -s /bin/bash postgres
bash-4.3$ whoami
postgres

If you fail to stop the indexer first, trying to drop the database results in the following:

bash-4.3$ dropdb mediaserver
dropdb: database removal failed: ERROR: database "mediaserver" is being accessed by other users
DETAIL: There is 1 other session using the database.

It does work perfectly though if the indexer is stopped. For reference, at some point I also did this, but it didn’t help:

super@DSM:/$ sudo synoservice -start synoindexd
super@DSM:/$ synoindex -R all

What was the last step? Rebooting the DSM. It took a while, but eventually the indexer recreated the database, the database started rebuilding and even though the Audio Station failed the first few times, suggesting the DSM was having issues, however with enough time the Audio Station behaved perfectly as expected. The correct directory became indexed, the information on the “Indexing Service” now referred to the correct application having access to the index, and the status was also updated correctly. It also took a bit longer for the Android app to work as expected, but eventually it allowed me to use the music from anywhere.

Search rankings for Google Mobile and ASK

Today I updated my Google search app on my phone. I had been monitoring the results on there for some time, but not in great detail. Looking at the first page only, the first hit remained – throughout this entire exercise – a link to the abandoned blog with my name listed on it. This has been the one big site which I have not been able to resolve either through outranking or through requesting content to be amended. It still has my name down as the owner of the site responsible for one of the email scams relating to Adobe and Skype products (as discussed previously, the scammers used both my stolen credit card details to pay and then used my name to register).

The update to the Google search app changed all that. Why? One little change in their app and algorithm – ranking locale specific to where you are, rather than the USA. This pushed the abandoned blog down to hit #9 immediately. A ranking I am much happier with compared to #1.

Similarly, if I use www.ask.com, that same problem site is #1, but If I use au.ask.com, it doesn’t appear at all!

Conclusions? It would seem that the only way I can get something to outrank that abandoned blog on US-based searches is to host things in a US-based way. That would be difficult, but may be necessary if I were to apply for a job in the US.

Latest Diaspora* update was a lot of work

So I’ve had some issues with Diaspora*, but the main one is inter-pod communication which doesn’t seem to work back to the main joindiaspora.com pod (and some others). I had neglected updating for a little while and thought I should get to it, and see if this issue was resolved.

**Caution – the rest of this discussion is fairly technical **

I had some trouble with the oath2 ruby gem for some time now as well, and during the update process I kept receiving errors about the .gemspec file format. I went through the usual update process to see if there would be an update to oath2 amongst others. So I used my usual 3 update commands:

git pull origin master
DB="mysql" bundle install --without development,test
RAILS_ENV="production" DB="mysql" bundle exec rake db:migrate

to find that it was more upset than usual about the update. The last command didn’t want to run at all, and there were far too many errors to see what was going on.

The easiest solution to this I thought would be to clean up the Ruby gems by updating everything and seeing what happened. To do a more thorough udpate of the gems, I ran

bundle update

which resulted in:

WARNING:  Invalid .gemspec format in '/usr/lib/ruby/gems/1.8/specifications/oaut
h2-provider-0.0.19.gemspec'
Gems included by the bundle:
Could not find oauth2-provider-0.0.19 in any of the sources

So I tried

gem update

and that’s when it all got interesting. Apparently I had run out of disk quote. Disk quota? As root? I check quota, sure enough, no quota applied. I ran a df to find I was using only 23% of disk space. So I tracked it down in the end – out of inodes.

The magic command

df -i

indicated that I’d used 100% of inodes. So then my search to find out where I could get more inodes from began.

It turns out you can clean up some old files from git by issuing a command to do a garbage collection:

git gc

of course, you can’t do this without having *some* spare inodes. So I cleaned up some otherwise unused files from a few places. Running this command gave me 25 spare inodes. Hmmm… still looking…. Then it turned out that you need to ask gem to do some housekeeping. Gem keeps all the previous versions of RubyGems, and isn’t shy of using a lot of files. The magic words for gem are:

gem cleanup

which gave me over 20,000 free inodes after much work, ignoring a lot of errors and saying no to any extra removals that it questioned. Much better! Then I could complete the gem update I tried earlier. Sadly, there were still a lot of errors and it made it difficult to see what was going on, so I went about how to fix the invalid .gemspec format error. Fixing this required an update to gem itself, which can be run with

gem update --system

another command which may be useful is

gem install rubygems-update

running this solved the .gemspec format error, but now 4 of the gems complained about an invalid datestamp format inside the .gemspec. This also needed fixing, which can be done by executing

gem pristine

and then updating back to current versions, followed again by cleaning up the unwanted files. You can do all of them at once, e.g. ‘gem pristine oath2 faraday mini_magick uuid’ although I should mention that one didn’t work the first time for me so I had to do that step again. Then you can run ‘gem update’ followed by ‘gem cleanup’ and hey presto – totally clean install of gems!

Now it was back in business with running the system properly. I ran the db:migrate successfully this time, and when I ran the server it couldn’t connect to redis. Apparently I had killed the process and shouldn’t have. Easily fixed with the old:

nohup redis-server &

So now diaspora* is running on my server again! Sadly, I still can’t communicate with other pods, but at least I got back in to the swing of system admin again.

Search rankings update

Following on from the two significant recent changes, and some other SEO work done, it is time to look at the rankings again. Also, I have let my ad run down again as most of the content has been amended, and my content is ranking reasonably well.

All of the engines in question now have updated the 37prime content. Interestingly, Google was almost immediate. Ask refreshed the content two days ago, and Bing/Yahoo! updated today. It has taken around the same amount of time to de-rank the now defunct Skype forum that also had the stolen identity related content (including some slanderous remarks). Google did take longer to de-rank than to refresh content, but the timing of the other two engines was concurrent with refreshing the content. Duckduckgo is still draging the chain on the content update.

www.google.com.au

Page 1: bad results at #5 (abandoned blog) (#7 is updated 37prime) (out of 10)

Page 2: none

Page 3: #6 (abandoned blog again)

au.yahoo.com

Page 1: none (out of 10) see bing below for more details

Page 2: #9

Page 3: #3

www.bing.com

Page 1: none (out of 10) – #9 is the updated 37prime content

Page 2: #9 (this is the abandoned blog)

Page 3: #3 (same blog – it is indexed multiple times because the same story can be linked to via different URLs based on the article tags. This phenomenon held true for the 37prime blog as well; the permalink and the archive both showed up)

www.ask.com

Page 1: #1 (the abandoned blog) (#5 is the updated 37prime post)

Page 2: #5 (same abandoned blog, through permalink)

Page 3: none!

duckduckgo.com

One continuous page: result #4 (37prime), 9 (Skype forum), 11 (abandoned blog) out of a total of 33 results. It looked rather different earlier this morning – 31 results, and the content had been updated, so their sources and algorithms appear to change.

So in summary, my SEO changes seem to have had some positive results for me on Bing/Yahoo! but haven’t made any difference to Google. Ask was slightly worse, and DuckDuckGo is a mixed bag. I will keep monitoring as I do want to de-rank the abandoned blog, as well as see what is happening on DuckDuckGo.

Same ol' spam, but different

So the same scam has been sent to me again (well, the Adobe version this time, not the Skype one). This scam has been around often and long enough (already back in 2010!, prior to them stealing my identity for one of the scam runs) that it has a Snopes page.

The updated spam, which I received a couple of days ago, again invents new versions of the adobe products that don’t exist. This time it’s Adobe 2012, and they are “charging” (i.e. stealing your details) for Adobe reader – which is always free.

This is what the scam looks like (and please don’t go to the links in the scam unless you know what you are doing) – although the mailout redirect goes through a third party, so the links have a landing page which collects usage information before sending you to the scam landing page. The scam landing page then takes you to the scam page proper that takes your money.

[sourcecode language=”html”]
INTRODUCING UPGRADED ADOBE ACROBAT READER 2012

Adobe is pleased to announce new version upgrades for Adobe Acrobat Reader

http://www.adobe-upgrade-2012.com

Advanced features include:

– Collaborate across borders
– Create rich, polished PDF files from any application that prints
– Ensure visual fidelity
– Encrypt and share PDF files more securely
– Use the standard for document archival and exchange

To upgrade and enhance your work productivity today, go to:

http://www.adobe-upgrade-2012.com

Start downloading the update right now and let us know what you think about it.

We’re working on making Adobe Acrobat Reader better all the time !

Copyright 2011 Adobe Systems Incorporated. All rights reserved.

Adobe Systems Incorporated
343 Preston Street
Ottawa, ON K1S 1N4
Canada
[/sourcecode]

So the main difference that I’ve noticed with the scam is that this time they’ve used some extra money (also presumably stolen) to pay for the site whois to not be shown:

Domain Name:adobe-upgrade-2012.com
Record created:11/29/2011
Record expired:11/29/2012


Domain servers in listed order:
	 ns1.dns-diy.net 	 ns2.dns-diy.net

Administrat:
   name-- Domain ID Shield Service
   org-- Domain ID Shield Service CO., Limited
   country-- CN
   province-- Hong Kong
   city-- Hong Kong
   address-- 1102-1103,11/F,Kowloon Bldg.,555 Nathan Rd.,Mongkok,Kowloon
   postalcode-- 999077
   telephone-- +852.22060092
   fax-- +852.30030133
   E-mail-- ad4561094151701@domainidshield.com
Technical Contact:
   name-- Domain ID Shield Service
   org-- Domain ID Shield Service CO., Limited
   country-- CN
   province-- Hong Kong
   city-- Hong Kong
   address-- 1102-1103,11/F,Kowloon Bldg.,555 Nathan Rd.,Mongkok,Kowloon
   postalcode-- 999077
   telephone-- +852.22060092
   fax-- +852.30030133
   E-mail-- ad4561094473302@domainidshield.com
Billing Contact:
   name-- Domain ID Shield Service
   org-- Domain ID Shield Service CO., Limited
   country-- CN
   province-- Hong Kong
   city-- Hong Kong
   address-- 1102-1103,11/F,Kowloon Bldg.,555 Nathan Rd.,Mongkok,Kowloon
   postalcode-- 999077
   telephone-- +852.22060092
   fax-- +852.30030133
   E-mail-- ad4561094473303@domainidshield.com
Registrant Contact:
   name-- Domain ID Shield Service
   org-- Domain ID Shield Service CO., Limited
   country-- CN
   province-- Hong Kong
   city-- Hong Kong
   address-- 1102-1103,11/F,Kowloon Bldg.,555 Nathan Rd.,Mongkok,Kowloon
   postalcode-- 999077
   telephone-- +852.22060092
   fax-- +852.30030133
   E-mail-- ad4561094539504@domainidshield.com

Despite this, it all does look very very similar and the one thing that cannot be hidden is the creation date of the URL. It is always a giveaway that if a website is contacting you unsolicited, and asks for your money from a web domain that has been put together in the last few days, they are up to no good.

Thanks to go out!

Thank you very kindly to 37prime for updating their post to link to my story of identity theft. On an extra happy note, the blog is clearly well indexed as the update came through on the Google search very quickly.

Search rankings update

Time again to do a snapshot. Since the last snapshot, I have submitted this blog to a number of blog indicies, as well as added some new content on other sites. Cross-linking between my own pages has also been updated. Finally, the negative content on the Skype discussion board has been retired, although still appears in search engines at the moment.

Same note as last time about Yahoo! and Bing. My Google ad is appearing periodically as they keep refunding small amounts of money based upon what they believe are bogus clicks. I’m really unclear on how this is assessed, and how they decide where to run the ads (apart from geographically – which is selectable).

www.google.com.au

Page 1: bad results at #5, #8 (out of 10)

Page 2: none

Page 3: #6

au.yahoo.com

Page 1: #8 (out of 10)

Page 2: #3, #6

Page 3: #2

www.bing.com

Page 1: #8 (out of 10)

Page 2: #3, #6

Page 3: #2

www.ask.com

Page 1: #1, #7 (exluding my ad which is running again!)

Page 2: #6

Page 3: none!

duckduckgo.com

One continuous page: result #3, 6, 9 out of a total of 32 results.

Ask is definitely on the improve, with the bad results sinking (apart from that abandoned blog at number 1). Google has gone in the right direction this time, but Yahoo!/Bing is definitely worse.

DuckDuckGo has also come out worse, but strangely now has a note about the results coming from Bing and being built by Yahoo!, but with no real explanation.

So is there much else I can do at this point without spending a lot of money? Having your online brand managed seems awfully expensive, and comes with little to no guarantees. As the Skype forum data disappears from primary searches, there will be some improvement but the other two sites are looking difficult to contain. There are no doubt other repositories of this data further down the search, and the Wayback Machine will no doubt have archived much of it, but I still would like to clean up the first 3 pages, and in particular, bump the retired blog from being the first hit on Ask.

Getting my Android ringtones back

This is a little post sharing my experience with getting my ringtones back on my HTC Desire Z (running official Gingerbread release).
**UPDATE 02-Dec-2011 This did not seem to survive a reboot. Worked when I copied the files across, but now the extra ringtones are no longer there.

I transferred my ringtones (in .m4a format) from my BlackBerry to Android phone some time ago. They worked perfectly. Came up in the menu. Everything was working. Then I decided to clean up the SD card to make some more space. In the cleanup I moved the ringtones off the phone. Most disappointed as I had made these myself.

After finding the files back, I thought I’d put them in a nice directory called

ringtones

under the

/media

directory. That turned out to do nothing, so I looked up where to put these files. Everywhere seemed to suggest just creating the

ringtones

directory (noting the lower case ‘r’) in the root of the SD card. That didn’t work either.

Then came the real clue – some advice to put them in a directory called

/media/audio/ringtones

So I created that directory (there was already a /media/audio/notifications but not a ringtones counterpart). I used the File Manager app to create the directory and move one of the ringtones there. Strange thing – once that directory existed, the phone found all of the .m4a files I had put there, so the other ringtones appeared as well.

So it seems that to use .m4a format ringtones, all you need is the directory /media/autio/ringtones to exist, and it will find other ringtones directories. Would need further testing, but the origninal ringtones from the first import were in /BlackBerry/ringtones and they worked perfectly. Odd behaviour overall. Would need some work to completely flesh out, but at least I have better understanding of where those files should be.

The distinction about .m4a files is that you can add any .mp3 media file as a ringtone, located anywhere on your SD card. There is a separate built-in add ringtone function for this.

Diaspora* update

After some server tweaking to make better use of resources, and some code updates, my Diaspora* pod is much more stable again. Although it is written in Ruby, the footprint isn’t too large – and I am yet to resort to low memory versions of Ruby.

I have now posted from Diaspora* to both Twitter and Facebook. The interface looks more polished with every update, and new little features and bugfixes are coming along nicely. It is still very much in the Alpha stage of development, but is improving every day. The link to Facebook seems to be much more stable now. This may have been a temporary developer Facebook issue, as it would not previously allow me to enter the correct details for security.

I would also like to take a moment to remember Ilya Zhitomirskiy, co-founder and a developer of Diaspora*, who left us far too soon.

Internet sense of humour

So a funny thing happened to me on the way to the internet. I’ve been monitoring where things are at, seeing if anything new comes up, and lo. A new search hit on ask.com. Apparently there’s enough data on me on the internet now to put me on hiprank.com. Naturally, I don’t rate very well, but it seems well enough to appear on hiprank.com. The thing I found most amusing was that the first thing that appeared (the ranks have changed again since a few hours ago) was:
russian hacker vs shawn sijnstra
which was surprising to say the least. I didn’t even know that my “hipness” was being measured, but there you go. And compared to an apparently mysterious russian hacker. While the issues happened on a Russian registrar, I don’t think I’ve ever identified the “hacker” as anyone in particular. I’m wondering whether it’s just an ad placed with an interesing piece of AI to generate the pages (there’s a lot of ads on that page), or if someone added this stuff manually? Perhaps the same kind of spammers that offer “comments” on this blog to link to their products.

Upon closer inspection, it appears that I’m compared to everything that has been related in this blog about my identity theft. Perhaps a semi-manual process has been used to generate pages, inflating the content on hiprank? Next question though, is how did hiprank suddenly appear on the first page of results on ask.com? They are scraping in to the bottom of page 3 on google.com.au, and top of page 5 on Bing. Oddly enough, duckduckgo doesn’t care. Props to them for working out ask’s ranking scheme.