Getting rid of spammers/sploggers (fake users) and their comments in WordPress

One of my other sites, Vision of Earth, has done well enough over the years that it has attracted attention from spammers. The problem I've been facing recently was that tens of thousands of "people" were subscribing to the site, receiving welcome emails, and attempting to post comments.

I use Akismet, so these comments weren't actually getting through to the blog itself, but their sheer number made them a problem.

First, auto-delete spam comments

This section assumes that you have the Akismet WordPress Plugin installed. Since it comes by default with most installations of WordPress, I think this is the easiest approach that is also very effective.

To make Akismet automatically delete (most) spam comments, go to Settings -> Akismet and click the option that says "Silently discard the worst and most pervasive spam so I never see it." Once you do that, Akismet will start deleting the vast majority of the spam you get. In my case, the number appears to be over 90%.

Akismet Spam Deletion Setting

If 90% isn't good enough for you, or you'd like assurance that every spam comment is deleted, I suggest that you use Spam Comments Cleaner. The setup is ridiculously simple and intuitive, and I can verify that it indeed works.

Second, stop spammers from registering

There are a variety of plugins out there that will help you with this. The one I decided to go with today was Stop Spammers. When you install it, you can optionally set it up to link up with StopForumSpam, Project Honey Pot, and BotScout via API keys (free). This was a pretty easy process, and one that I'd recommend for the future health of the Internet. By linking up with these tools, we both empower them and empower ourselves.

Third, clean spammers/sploggers out of the database

Warning: I didn't find an easy way to do this. The plugins WangGuard and Inactive User Deleter purport to be able to automatically delete these accounts. I tried both of them and they both failed. For my tests, I had about 46 thousand inactive/spam accounts on my server.

WangGuard can't batch delete more users than you can view at one time on the "Users" menu in WordPress. If I raise the number of viewable Users (by clicking "Screen Options" in the top-right) to more than a few hundred, the plugin fails with an out-of-memory problem.

Inactive User Deleter purports to be able to filter users, but it's selective power is very weak and it also crashes with an out-of-memory error when I try to run it.

I, unfortunately, had to do this manually. Luckily I know what I'm doing in a MySQL database and was able to find enough help online to finish the task. If you want to follow the rest of this guide, you'll need to have command-line access to your server. Similar actions might be possible on various server management systems, but I won't deal with them here.

Before I could start deleting accounts, I needed to be absolutely sure that I was going to be able to:

  1. recover from this effort in case it fails and
  2. delete only those Users that I specifically want to delete.

Make a backup of certain database tables

On the command line, I ran:

mysqldump <wordpress_database_name> wp_users wp_usermeta -u <wordpress_username> -p > ~/2014-10-05-database_cleanup_backup_of_users_and_usermeta.sql

The intent of this command is to backup the wp_users and wp_usermeta tables from the <wordpress_database_name> database. For me this created a 28 megabyte file. Note that the '<' and '>' symbols shouldn't be present in your command (except for the final one after '-p').

If you don't know your <wordpress_database_name> or <wordpress_username> (or its associated password, which you'll need to type in after you run the command), you can find them in the wp-config.php file in the base directory of your WordPress installation.

You will see lines like:

define('DB_NAME', 'mydatabasename'); // The name of the database
define('DB_USER', 'mydbusername'); // Your MySQL username
define('DB_PASSWORD', 'mypassword123abc'); // ...and password

You'll need all of these pieces of information in the next section as well.

Note that you can restore these tables by running:

mysql <wordpress_database_name> -u <wordpress_username> -p < ~/2014-10-05-database_cleanup_backup_of_users_and_usermeta.sql

Beware, if you do this at a later date, you'll destroy any information that was added to these tables since you made the backup. This is a backup that you don't want to have to use in the distant future. You can use more advanced MySQL queries to load these things without losing what you have, but that is far beyond the scope of this post.

Manually delete the bad user accounts

I log into mysql using this command:

mysql <wordpress_database_name> -u <wordpress_username> -p

You'll need to enter the password to get in. Once in, I tested a bunch of queries to ensure that I understood how many users existed and how many I wanted to delete, and of what types. In my case, I wanted to delete everyone who was a "Subscriber" who had not created a comment or a post. These are pretty intense criteria. Don't blindly apply what I did here unless you want to delete exactly that set of people.

I deleted the user entries:

delete from wp_users WHERE ID NOT IN (SELECT post_author FROM wp_posts UNION SELECT user_id FROM wp_comments UNION select distinct user_id from wp_usermeta where meta_key = 'wp_capabilities' and meta_value not like '%subscriber%' );

When the command is successfully completed, it will tell you something like:

Query OK, 45421 rows affected (8 min 36.24 sec)

Now to get rid of the records in wp_usermeta, we run:

delete
FROM wp_usermeta
WHERE user_id NOT IN (SELECT ID FROM wp_users);

When it is successful, it will show you the number of rows deleted, like so:

Query OK, 547468 rows affected (20.43 sec)

Note that wp_usermeta will probably contain more than ten times as many entries as wp-users.

And, you're done. Your WordPress interface should update with these new numbers. Congrats, you've cleaned out all the inactive / spammer user accounts on your server.

Happy blogging.

Turn off touchpad while typing on Linux

I installed Linux Mint 16 a few days ago on my Netbook. The touchpad didn't turn off while I was typing, so I dug around for a way to make it do so.

It turns out you can enable this feature on the "Mouse and Touchpad" configuration app (just search for that string among your applications) on the Touchpad tab. However, this default setting didn't work for me, because it turned off the touchpad for two seconds every time I typed something. For me, that was way too long.

(Note, in the following setup, I set my delay to 0.25 seconds. Feel free to change the 0.25 in the command in step 5 to whatever you want.)

Thus, it was a more manual configuration for me. Here goes:

  1. Run "Session and Startup". Again, you can just search for this from the "Start Menu" for Linux Mint.
  2. Go to the tab "Application Autostart".
  3. Click "Add"
  4. Type a name like "Synaptic Daemon", add in some comments about what you are doing in the description field, like: "Turn of touchpad while typing"
  5. In the command window, put:

    syndaemon -d -i 0.25 -K -R

  6. Click OK
  7. Make sure that the entry you just created has a little "x" in the box to the left of it.
  8. Click close.
  9. Restart your system.

Now your touchpad should not accept input while you are typing. Almost immediately after you stop typing, it should again respond to your movements.

Changing the order of entries in Grub 2 bootloader

A while back I had to solve the problem of removing an operating system entry from the Grub 2 bootloader.

Today I solved a related problem: changing the order in which the OS entries appear on the boot menu. It seems there are several solutions to this problem. The one I will show here was the most effective for me because I have already used the approach in the link above to remove an OS from Grub 2.

Here's how you do it:

  1. Log in as root, or execute the following mv commands using "su" before your commands.

    sudo su

  2. Go to the "/etc/grub.d" directory.

    $ cd /etc/grub.d

  3. Look at the directory:

    $ ls

  4. Which will give you a result that look something like this (numbers and names may vary).

    00_header 10_linux 30_os-prober 41_custom
    05_debian_theme 20_linux_xen 40_custom README

  5. Basically, the order of these numbers dictate what the order will be on the Grub menu. You can rename these files to change the order. In my case, I wanted to make one of my "custom" entries the first entry (because it was my Windows 7 partition).

    $ mv 10_linux 15_linux
    $ mv 40_custom 10_custom

  6. Now I just re-run update-grub to re-generate the boot menu table.

    $ update-grub

  7. We can check that things are ok by looking through the generated file. In my case, I checked that my Windows 7 entry was present in the file. Interpreting this file deeply requires a lot of knowledge, so don't worry if it is mostly gibberish.

    $ vi /boot/grub/grub.cfg

  8. Restart your computer to see the new boot menu order.

Experimental Review of Graphene

My first academic paper was officially published as of the beginning of this year.

Experimental Review of Graphene (pdf)

Experimental Review of Graphene
Daniel R. Cooper, Benjamin D’Anjou, Nageswara Ghattamaneni, Benjamin Harack, Michael Hilke, Alexandre Horth, Norberto Majlis, Mathieu Massicotte, Leron Vandsburger, Eric Whiteway, and Victor Yu
ISRN Condensed Matter Physics
Volume 2012 (2012), Article ID 501686, 56 pages, doi:10.5402/2012/501686

The full-text paper is available as html, and as a pdf by ISRN. A preprint (with chapter headings) is available on arxiv (full-text pdf).

The review is intended to provide an introduction to the study of graphene from an experimental perspective. The topics covered within are:

  1. Electronic structure
  2. Vibrational properties (phonons)
  3. Synthesis (fabrication)
  4. Characterization (measurement / detection techniques)
  5. Electronic transport and field effect (scattering, mobility, conductivity)
  6. Magnetoresistance and the quantum Hall effect
  7. Mechanical properties (micromechanical oscillators, actuators)
  8. Graphene transistors
  9. Optoelectronics (transparent conducting electrodes, photodetectors, light-emitting diodes, photovoltaics, quantum dots)
  10. Sensors (electrochemical and biosensors)

Heaven Seven Video from Mekka & Symposium 2000

In 2000, a sizable group of people got together at a conference called Mekka & Symposium. Their artform, known as 'Demos', continues to blow my mind to this day. The idea is that you write a tiny computer program that is capable of generating an incredible series of visual and auditory stimuli. The result is generally a cohesive 'video', that we today can watch via systems like YouTube.

For me, the really incredible part of this effort was the fact that they would compete to see who could do the most with a program of a given size. The most spectacular achievements on this front that I am aware of were all programs at around 64 kilobytes in size. In order to put that in perspective, all of the information in the program file would be about 64,000 characters, or about 10,000 words in a human language. These programs demonstrate some of humanity's absolutely stunning creativity and ingenuity.

One of my favourite videos of all time, Heaven Seven, is one such Demo. Check it out (in full 1080p) on YouTube through the link below:

Alternately, you can still get the original executable version of Heaven Seven.

Dual wielding laptops

My poor netbook (Samsung NF210, which you might remember from my issue with the Grub 2 menu) has been dying under the weight of Matlab, LabView, and all the other normal programs that I run on a constant basis such as Acrobat Reader, TeXnicCenter, and Chrome.

I have had to pull my old laptop (Acer Aspire 5670) out of retirement. After a couple hours of setup, I had two laptops set up on my desk. This is when I found out that I am not an embarrassingly parallel procedure. By effectively doubling the number of processors that I was running on, my productivity went up about 50%.

The key to all of this has actually turned out to be a piece of software that I admit I turned my nose up to a bit when I first saw it...

Dropbox

I installed Dropbox on both laptops and told them to sync my folder full of work files. A couple minutes later everything was mirrored on both computers as well as on the online system.

Since I have wireless Internet on both laptops, they can both connect instantaneously the the online Dropbox repository and sync files with one another. The total lag time from when I save a file on one to when it shows up on the other is on the order of mere seconds.

This allows me to work on highly technical material on one computer, and be completely confident that all of my progress is saved elsewhere. Additionally, it gives me great mobility and freedom of mind.

I admit, I should probably be a poster child for Dropbox at this point.

Care about your files? Not too worried about putting them in the cloud? Try Dropbox. It impressed me.

Matlab Fast-Fourier Transform (FFT)

I have spent a lot of time in the last couple weeks fighting to discern some meaning from the results of some Fourier transforms I was doing. During this time I ran across some very helpful resources.

First and foremost among them is this excellent FFT tutorial for Matlab over at blinkdagger. I found this code and explanation to be very useful.

I still had a lot of learning to do however, especially about the correct frequency axis for my data. The simplest way to understand it (as far as I know) is this:

If you run an FFT on 9000 real-valued data points, then it will return an array of 9000 complex values. If you plot the complex modulus (absolute value, or abs(data) function in Matlab), then you will see that the magnitude of the data is symmetric. That is, the first 4500 points are the same as the last 4500 points. In fact there is an interchange of complex and real parts of the symmetric data, but we won't get into that here.

The important part for me was to realize that if I look at only the first 4500 points, then point number 1 represents the lowest frequency component in the signal that can be detected. The 4500th point is the highest-frequency point that can be detected. What frequency is this point?

It is the Nyquist frequency, equal to half of the sampling rate. The maximum frequency that we can extract from the data is half the sampling frequency. Therefore if your data was taken at a rate of six per second (6 Hertz, or one every 0.166667 seconds), then the Nyquist frequency would be 3 Hertz.

This allows you to create a meaningful frequency axis for your Fourier-transformed data. The highest point is the Nyquist frequency, and the lowest point would be equal to the Nyquist frequency divided by the number of data points. In my above example, that would be 3 Hertz / 4500 data points = 0.00066666 Hertz.

As simple as this sounds, it did take me a little while to figure out.

Secondly, in case anyone is taking the FFT of real-world data, it is very likely that you will have to view it on a log-log axis in order to get anything meaningful out of looking at it. Much of the useful information to humans is often clustered around the low-frequency end. Viewing the data in log-log is very helpful for this. Also, this view can help you identify intensity dropoffs with increasing frequency. Some common ones are 1/f and 1/f^2.

To view in log-log, the matlab command is:

loglog(xAxisData,yAxisData);

If there are no other figures created, this will create a figure with your plot in it.

Manually removing an OS from Grub 2 bootloader in Ubuntu 11.10

This post is here partially because I have had to go through this action several times in the last few months.

I recently bought a Samsung NF-210 Netbook and I have been fighting with setting it up since then. The netbook comes preloaded with Windows 7 Starter. I added in Ubuntu as a dual-boot option. I started with Ubuntu 10.10, but that proved to be difficult to use because it did not correctly handle the function keys for changing brightness or volume.

This same solution worked on Ubuntu 10.10 as 11.10.

My goal is to remove the Windows Recovery Environment from the automatic listing on the grub boot menu. This is more than just a cosmetic change because if you accidentally run this partition, it will fubar your boot partition causing you to lose the ability to boot using grub. Thus you will have to reinstall grub or possibly an entire OS (either linux or windows) depending on how you have set up your hard drive.

  1. Fire up a terminal, switch to root via "sudo su".
  2. "cd /etc/grub.d" - changing to the directory where we do most of the work.
  3. "chmod -x 30_os-prober" - turning off the automated ability of grub to find new non-linux OSes.
  4. "vi /boot/grub/grub.cfg" - going to see the current profile of OSes detected on the computer.
  5. Scroll down until you spot the entries for your windows partitions. Select the one(s) that you do want to show up on the menu. Copy them to the clipboard. Copy this text to somewhere so that you can paste it into the next file...
  6. "vi 40_custom" - This custom file lets you add your own OS profiles to the grub menu.
  7. Paste in the text that you copied earlier (below the comments).
  8. "chmod +x 40_custom" - The file needs to be executable in order to be included in the grub menu.
  9. "update-grub" - Grub needs to re-create its menu.

Now when you reboot you should see that only the OSes that you selected and moved will show up. In my case I moved my Windows 7 profile, but not the Recovery Environment profile. Thus when I boot I see all of my menu items except the recovery profile. Yey!

Musings inspired by Ashoori's article "Electrons in Artificial Atoms"

These notes come primarily from reading an article by R.C. Ashoori of MIT which was published in the journal Nature, Volume 379, February 1996. The article is entitled "Electrons in Artificial Atoms".

In an artificial atom, the effects of electron-electron interaction are more important than in a normal atom! This is because orbital energies are far lower in artificial atoms than in real ones. The opposing effect of the spreading out of the electrons in space is not as large. Thus the relative importance of electron-electron interactions increases.

Energy resolution of new spectroscopic techniques is only limited by the sample's temperature.

Ashoori describes a setup where a QD is close enough to one contact (capacitor plate) that an electron would be able to quantum tunnel between them. The other capacitor plate is too far away to tunnel. When an electron is successfully added to the QD, you can detect a tiny change in charge (typically about half an electron charge) on the surface of the farther capacitor plate.

A neat extension of this idea is to attach an AC current to the DC gate voltage that is driving electrons onto the QD. This makes it possible for an oscillation to occur at specific DC gate voltages. This is when the electron tunnels to and from the QD with each oscillation of the AC part of the gate voltage. This allows for synchronous detection by the farther capacitor plate as its charge changes slightly in time with these oscillations. This is known as single-electron capacitance spectroscopy (SECS).

Gated transport spectroscopy (GTS) seems to be what we do with our double quantum dots. We maintain a voltage difference between the source and drain contacts, and thus we can measure the current flow changing with changing conditions such as a changing gate voltage. When this article was written (1996), no one had yet successfully conducted GTS with fewer than about 10 electrons on the dot.

There are two main effects that make it more difficult to add extra electrons to a QD. The first is electron-electron interactions. They obviously push each other away. This is called the charging energy. Then there is the quantum energy levels. In order for an electron to be present in the dot, it needs to be occupying a quantum level. Due to the Pauli exclusion principle, it is not possible for more than two electrons to occupy the exact same quantum level. The factor of two is due to different possible spins of the electrons. This review claims that the charging energy is about five times the quantum level spacing for the samples described in the paper.

There is a geometric factor that connects the value of the gate voltage with the actual amount of energy needed to add an electron to the dot. This paper seems to be claiming that they simply use the geometry of the sample to estimate this. In the case of their perfectly symmetric doubly-contacted QD, they claim that this geometric factor is 0.5.

Modeling

The author sketches out the basics of the parabolic potential well assumption. They assume that the z-direction is completely constrained, and that the x and y directions are governed by the parabolic potential well. The potential is circularly symmetric.

I have also read elsewhere that this model matches up rather well with observations. Even in 1996 this was apparently already known.

Introduction of a constant magnetic field to the analysis breaks the degeneracy in the quantum number l. Now positive and negative l's have have slightly different energies due to the contributions of the magnetic moment interactions with the magnetic field. Note that we are talking about the magnetic moment created by the evolution of the electron's wavefunction such that the electron can be considered to be moving in a circle around the center of the potential well. Magnetic field applied along the z axis enhances confinement in the dot. The magnetic field also introduces Zeeman (spin) splitting due to the magnetic moments of the electrons.

With the introduction of the magnetic field to the discussion, the author began to refer to the quantum levels as Landau levels. Strong magnetic fields can cause only one side of the level, let's say the side with positive l, to be populated.

There is an interesting plot demonstrating the zig-zag effect as electrons end up populating different levels as the magnetic field strength increases. At some point, the zig zag stops because all electrons are in the lowest Landau level (and on one side of the l range I believe). This is very striking when seen in a plot.

Another important effect of increasing magnetic field is the fact that all of the radii of the different l levels shrink. This makes sense in light of the observation we made above that the magnetic field increases the confinement strength.

Example with 2 electrons in ground singlet state (l = 1, s = +-1). With increasing B-field, it is possible to increase the Zeeman energy enough that one of the electrons gets promoted to l = 1.

Interactions of magnetic moments alone is not sufficient to produce the observed spin flips and l transitions. We must take into account coulomb interaction between the electrons in order to get the right answer. As the magnetic field increases, the l radius decreases and eventually it becomes possible for an electron to jump into the l = -1 state from the l = 0 state.

Once in the lowest Landau level, more changes can still occur. The lowest energy of all the electrons if they are in the lowest Landau level would nominally be when they are paired off with each pair having opposing spins to each other. As the magnetic field increases however, eventually higher-l states become lower-energy than either the spin up or spin down states, depending on the direction of the magentic field. This means that we will continue to see bumps in the spectrum as electrons will flip spins and move to more distance l values. Self-consistent calculations can reproduce some of these effects, but they seem to overstate the number of flips that happen at low field, and underestimate the number of flips at high field. Something is obviously still missing.

The Hartree-Fock technique takes into account the repulsion between many different electron wavefunctions. It reproduces much of the correct behaviour. An interesting result is that there is actually a short-range attraction force acting on electrons that are in different l bands. Adjacent bands tend to be preferentially populated. This can be compared to the fact that electrons in the same l band tend to be on opposite sides of the dot from one another. By moving to the higher l bands, it seems that the electrons can both be in a lower energy state and be closer to one another, an apparent paradox.

When the magnetic field becomes even higher, eventually it becomes energetically favourable for gaps to form in the l spectrum. The lowerest energy states involving gaps typically involve the gaps being adjacent to one another. Thus we don't end up with a smattering of l gaps throughout our states. We end up with one big block of empty l's.

It is known however that the Hartree-Fock calculations leave something out. They do not take into account the known electron correlation. In this area, some other techniques such as "exact diagonalization" seem to be better. However, the authors do not mention a successful combination of these factors into one theoretical model.

Their final section mentions some exciting further work that is being pursued, or will likely be pursued, in the field of quantum dots. One of the more interesting points for me is that a single-electron transistor obviously has a 'fan-out' problem. That is, normally the result of a piece of digital logic can be used to set off a cascade of other logic operations. This is obviously difficult if the result of your digital logic operation is the movement of a single electron. It seems however that people are finding ways around this. Perhaps I will find out more about this in the future.