A set of positive integers $A\subset \mathbb N$ is called a Sidon Set or a Sidon Sequence if the equation $a+b=c+d$ does not have any non-trivial solutions in $A$. They were named after Hungarian mathematician Simon Sidon who was inspired by certain problems in Fourier series to ask Erdős about the possible growth of such sequences.
Since then, there has been an extensive amount of work on this topic exploring a plethora of different questions about finite and infinite Sidon sets, and various generalizations of them. In this post, we are mainly concerned about a breakthrough theorem in the field of infinite Sidon sets.
First consider the following example.
Example : (Greedy Algorithm) If $S\subset [1,n]$ is already a Sidon set, we can find $x$ such that $x$ does not equal $a$, $b$, $c$ or $a+b-c$ for any $a,b,c\in S$ and $S\cup \{x\}\subset \mathbb N$ is Sidon. Since there are at most $n$ choices for each of $a$, $b$ and $c$ and $a,b,c\le n$, choosing $x>n^3$ is enough.
It is clear that this sequence satisfies $S(x)\gg x^{1/3}$ where $S(x) := \left\lvert S\cap [1,x] \right\rvert$. It was conjectured by Erdős that there is a Sidon sequence $S$ with $S(x)\gg x^{1/2-\varepsilon}$ for all $\varepsilon>0$ (recall that $f(x)\gg g(x)$ means that there exists a constant $C$ such that $f(x)\ge C g(x)$ for all large $x$ and $f(x)\ll g(x)$ is defined similarly). He also proved that any arbitrary Sidon sequence will have the close upper bound of
$$S(x)\ll x^{1/2} \left(\log x\right)^{-1/2}$$
for infinitely many $x$. This exhibits that finite segments of an infinite Sidon set must sometimes be much smaller than it is expected in finite Sidon sets.
The lower bound was first improved by Ajtai, Komlós and Szemerédi who proved the existence of $S$ satisfying
$$S(x)\gg x^{1/3}\left(\log x\right)^{1/3}$$
and then finally Ruzsa gave a breakthrough by finding another $S$ satisfying
$$S(x)\gg x^{\sqrt 2 -1 - o(1)}$$
which has remained the best result in this area since $1998$. It is this proof that I will present in this post.
For those asking why we care, it is known that the number of elements in the biggest (usually called dense in literature) Sidon subset of the interval $[1,x]$ is of the order $\sqrt x$ (see here) and what Erdős' conjecture claims is that $\mathbb N$ is quite similar. In other words, we can find Sidon subsets of $\mathbb N$ with (logarithmic) density arbitrarily close to $\sqrt x$ (although $\sqrt x$ is not achievable, and this was proven by Erdős).
Theorem : There exists an infinite Sidon sequence $B$ such that
$$B(N)=N^{\gamma+ o(1)}$$
where $\gamma=\sqrt 2-1$.
Idea of the Proof : Notice that the set of primes $\mathcal P$ form a multiplicative Sidon set, that is $p_1p_2=p_3p_4$ for $p_i\in \mathcal P$ implies $\{p_1,p_2\}=\{p_3,p_4\}$. So, $\{\ln p\}_{p\in \mathcal P}$ forms a Sidon set in $\mathbb R$.
This means that if we can cut the binary representation of $\ln p$ in a proper way, we still almost surely retain the Sidon property. In fact, all the observations we made so far are true for $\{\alpha \ln p\}_{p\in \mathcal P}$ for all $\alpha$. Indeed we will start with a bunch of $\alpha$ so that we can later argue probabilistically.
Proof Setup : The first step is to look at the number $\alpha\ln p$ in binary. Choose $\alpha\in [1,2]$ and write
$$\alpha \ln p = \sum_{i=0}^{k_p}\varepsilon_{ip} 2^i + \sum_{i=1}^\infty \delta_{ip} 2^{-i}$$
where $k_p=\left\lfloor\log \left (\alpha \ln p\right )\right\rfloor$. So, the $\varepsilon$-string is the integral part of this number and the $\delta$-string is the part after the decimal point (or is it the binary point?). Here, and in the rest of the proof, $\log$ will represent $\log_2$ and $\ln$ will be a placeholder for logarithm to any pre-specified base - the reader can think of it as $\log_e$ for convinience.
Now, we set $\beta = \sqrt 2 +1$ and $\gamma=1/\beta$. Then, we define
$$K_p = \min \left\{i>2 : 2^{(i-1)^2}>p^\beta\right\}$$
so that $K_p=2+\lfloor \sqrt{\beta \log p}\rfloor$. A more natural way to present this proof could have been to start with an arbitrary $\beta$ and realize at the end of the proof that this is the value of $\beta$ that optimizes the density.
Now, we need to do the proper cutting. So, we define
$$\lambda_p = \sum_{i=0}^{k_p} \varepsilon_{ip} 2^i + \sum_{i=1}^{K_p^2} \delta_{ip} 2^{-i}$$
by cutting the binary expansion of $\alpha \ln p$ at the $K_p^2$-th place.
Now, we use the square indices to construct
$$\Delta_{ip}=\sum_{j=(i-1)^2+1}^{i^2} \delta_j 2^{i^2-j}$$
by cutting the bigger block into smaller pieces.
In binary, the numbers look like
\begin{align}\Delta_{1,p} &= \delta_1\\\Delta_{2,p} &=\delta_2\delta_3\delta_4\\\Delta_{3,p} &=\delta_5\delta_6\delta_7\delta_8\delta_9 \end{align}
and so on. From now on, we will often supress $p$ or $\alpha$ from our notation whenever it doesn't interfere with the clarity of the proof.
Finally, we define $b_p$ as
$$\dots \underline{\smash{\:0\:}}\;\underline{\smash{\:\textcolor{red}{1}\:}}\;\underline{\smash{\:0\:}} \;\Delta_K \dots \Delta_3\; \underline{\smash{\:0\:}}\;\underline{\smash{\varepsilon_2}}\;\underline{\smash{\:0\:}}\;\underline{\smash{\:0\:}}\;\underline{\smash{\:0\:}}\; \Delta_2 \;\underline{\smash{\:0\:}}\;\underline{\smash{\varepsilon_1}}\;\underline{\smash{\:0\:}}\;\underline{\smash{\:0\:}}\;\underline{\smash{\:0\:}} \;\Delta_1 \;\underline{\smash{\:0\:}}\;\underline{\smash{\varepsilon_0}}\;\underline{\smash{\:0\:}}\;\underline{\smash{\:0\:}}\;\underline{\smash{\:0\:}}$$
which was done in the following way : first write the $\Delta_i$'s in a piece of paper and put five blanks in between each of them, and after $\Delta_1$. Fill up the second blank to the right of $\Delta_i$ with $\varepsilon_{i-1}$. The $4$th blank always contains a $0$ except the one to the left of $\Delta_K$ - this is done to keep track of the size of $b_p$. The rest of the blanks are all $0$.
More formally,
$$b_p = \sum_{i=1}^K \Delta_{ip}2^{(i-1)^2+5i} + \sum_{i=0}^k \varepsilon_{ip} 2^{i^2+5i+2} + t_p$$
where $t_p=2^{K^2+5K+4}$.
Further, we put $\Delta_{ip}=0$ for $i>K$ and $\varepsilon_{ip}=0$ for $i>k$. Our Sidon set will be a subset of the set $\{b_p\}_{p\in \mathcal P}$.
Plan of the Proof : Recall how we introduced a padding of $0$'s in the definition of $b_p$. This was done to ensure that there are no carry overs when we add $b_{p_1}+b_{p_2}$. This makes sure that
\begin{align}
b_p + b_s = b_q + b_r \iff & t_p + t_s = t_q + t_r, \\ & \varepsilon_{ip} + \varepsilon_{is} = \varepsilon_{iq} + \varepsilon_{ir}, \\
& \Delta_{ip} + \Delta_{is} = \Delta_{iq} + \Delta_{ir}
\end{align}
for all $i$. Now, consider
$$b_p+b_s=b_q+b_r,\qquad b_p>b_q\ge b_r>b_s \tag{*}$$
for some $p,q,r,s$.
Define $P_K=\{p : K_p=K\}$. These sets provide a proper partitioning of the primes with respect to their sizes, and hence have the following important property. If $(*)$ holds, then there are integers $K\ge L$ such that
$$\lambda_p+\lambda_s = \lambda_q+\lambda_r \tag{1}$$
and $p,q\in P_K$, $r,s\in P_L$.
The artistic part is now over and what's left of our work is the technical part. This is the part where we try to filter out the largest Sidon subset of $\{b_p\}_{p\in \mathcal P}$. The techniques are all mainly aimed towards estimating the number of solutions of $(*)$ in the set $\{b_p\}_{p\in \mathcal P}$.
Bounds : Notice that
$$\left\lvert\lambda_p-\alpha\ln p\right\rvert\le 2^{-K^2}<p^{-\beta}$$
follows from definitions. Also, if $(*)$ holds, then
$$\left\lvert ps-qr\right\rvert < 8qr\cdot 2^{-L^2},\qquad qr > 2^{L^2-2}, \qquad (K-1)^2 > (\beta-1)(L-1)^2 + \beta(2L-3)$$
where $K$ and $L$ are as in $(1)$.
Now, we define
$$J_{KL} = \left\lvert\left\{p,q,r,s : p\neq q\in P_K, \; r\neq s\in P_L,\; \left\lvert ps-qr\right\rvert<8qr\cdot 2^{-L^2}\right\}\right\rvert$$
and try to estimate it.
Lemma : We have
$$J_{KL}\ll 2^{2\gamma \left((K-1)^2+(L-1)^2\right)-L^2}$$
where $J_{KL}$ is defined as above.
Proof : The Diophantine equation $ps-qr=a$ has the general solution
$$p=p_0+tr,\qquad q=q_0+ts$$
where $p_0$, $q_0$ are the smallest solution.
As $p<2^{\gamma(K-1)^2}$, we have $t<\frac{2^{\gamma(K-1)^2}}{r}$. So, for fixed $a,r,s$, the number of solutions is $\mathcal{O}\left(\frac{2^{\gamma(K-1)^2}}{r}\right)$.
Again, $\left\lvert a\right\rvert \ll r\cdot 2^{\gamma(K-1)^2-L^2}$. So, for fixed $r,s$, the number of solutions is $\mathcal O\left(2^{2\gamma (K-1)^2 -L^2}\right)$ and $r,s<2^{\gamma (L-1)^2}$.
Lemma : We have
$$\left\lfloor 2^{K^2}\alpha \ln p\right\rfloor \equiv \left\lfloor 2^{K^2}\alpha \ln q\right\rfloor \pmod{2^{K^2-L^2}}$$
assuming $(*)$ holds.
Proof : It is clear that
$$\left\lfloor 2^{K^2}\alpha \ln p\right\rfloor \equiv \sum_{i=L^2+1}^{K^2} \delta_{ip}2^{K^2-i} \pmod{2^{K^2-L^2}}$$
from definition. Also for $L<i\le K$, we have $\Delta_{ip}=\Delta_{iq}$ so that $\delta_{jp}=\delta_{jq}$ for $L^2<j\le K^2$. So, RHS remains unchanged if we replace $p$ by $q$.
Lemma : Let $K>L$ and $p\neq q\in P_K$ be given. Let there be at least one pair $r,s\in P_L$ and an $\alpha\in [1,2]$ such that $(*)$ holds. Then, we have
$$\mu\left(\alpha : \left\lfloor 2^{K^2}\alpha \ln p\right\rfloor \equiv \left\lfloor 2^{K^2}\alpha \ln q\right\rfloor \pmod{2^{K^2-L^2}}\right)\ll 2^{L^2-K^2}$$
where $\mu$ is the Lebesgue measure.
Proof : Call $A=2^{K^2}\ln \frac pq$, $M=2^{K^2-L^2}$ and notice that the property we are after is
$$\left\lfloor A\alpha \right\rfloor \equiv 0 \text{ or } -1 \pmod{M}$$
and since $\left\lfloor A\alpha \right\rfloor$ is evenly distributed modulo $M$, we have
$$\mu\left(\alpha : \left\lfloor A\alpha \right\rfloor \equiv 0 \text{ or } -1 \pmod{M}\right)\ll \frac 1M$$
hence completing the proof.
The proofs of the next two lemmas are straightforward and hence we will refrain from exhibiting them.
Lemma : For $p\neq q\in P_K$ and $r,s\in P_L$, we have
$$\mu\left(\alpha : b_p+b_s = b_q+b_r\right)\begin{cases}=0 &\text{if} \left\lvert ps-qr\right\rvert\ge 8qr\cdot 2^{-L^2}\\ \ll 2^{L^2-K^2} &\text{if} \left\lvert ps-qr\right\rvert <8qr\cdot 2^{-L^2}\end{cases}$$
for $K>L$.
Lemma : If
$$G_{KL}(\alpha) := \left\lvert\{\;p,q,r,s : p\neq q\in P_K, r,s\in P_L, (*) \text{ holds }\}\right\rvert$$
then
$$\int_1^2 G_{KL}(\alpha) \;\text{d}\alpha \ll 2^{2\gamma\left((L-1)^2+(K-1)^2\right)-K^2}$$
for $K>L$.
Lemma : Define
$$G_{K}(\alpha) = \left\lvert\{\;p,q,r,s : p,q\in P_K, (*) \text{ holds }\}\right\rvert$$
so that
$$\int_1^2 G_{K}(\alpha) \;\text{d}\alpha \ll 2^{\gamma\left(K-1\right)^2-2K}$$
for all $K$.
Proof : By definition, we have $G_K(\alpha)=\sum_{L\le K} G_{KL}(\alpha)$. And since $G_{KL}(\alpha)\neq 0$ only if $(L-1)^2<(K-1)^2/(\beta-1)$, we obtain an estimate of $\ll 2^Z$ with
$$Z=\left(\frac 2{\beta -1}-1\right)(K-1)^2 -2K +1$$
on which we use the fact that $\frac 2{\beta -1}-1=\frac 1\beta$.
In fact, the value of $\beta$ was chosen in the first place to satisfy $\frac 2{\beta -1}-1=\frac 1\beta$.
Proof of Main Theorem : With all the setup in hand, the rest of the proof is essentially an exercise in probability theory. Define the event
$$E_K := \left[G_K\ge N_K\right]$$
where $N_K:= 2^{\gamma(K-1)^2-K}$. This gives
\begin{align*}\mathbb P(E_K) &= \mathbb P(G_K\ge N_K)\\&\le \frac{\mathbb E(G_K)}{N_K}\;\;\text{(using Markov Inequality)}\\&= \frac 1{N_K} \int_1^2 G_K(\alpha)\text{d}\alpha\\ &\ll 2^{-K}\end{align*}
and hence
$$\sum_{K} \mathbb P(E_K) < \infty \implies \mathbb P\left(\liminf_{K} E_K^C\right)=1$$
by Borel-Cantelli lemma.
This guarantees the existence of an $\alpha$ such that
$$G_K(\alpha) < 2^{\gamma (K-1)^2 - K}$$
for all (except at most finitely many) $K$. Fix this $\alpha$.
Using Prime Number Theorem, the cardinality of $P_K$ is
$$\left\lvert P_K\right\rvert = \pi \left(2^{\gamma (K-1)^2}\right) - \pi \left(2^{\gamma (K-2)^2}\right) \sim \frac{2^{\gamma (K-1)^2}}{\gamma K^2 \ln 2}$$
and hence $G_K(\alpha)<\left\lvert P_K\right\rvert /2$ for $K>K_0$.
Now, for each $p,q,r,s$ satisfying $(*)$ with $p\in P_K$, we omit $p$ and define $Q_K$ to be the set of remaining elements. Then we define
$$B = \bigcup_{K>K_0} Q_K$$
which, by definition, is a Sidon set.
Now, we have
$$b_p < 2^{K^2+5K+5} < 2^{\left(K+3\right)^2}$$
implying that for $K=\left\lfloor \sqrt{\log x} -3\right\rfloor$, $\max Q_K <x$. This gives
$$B(x)\gg \left(\frac 12 -\varepsilon\right) \pi\left(2^{\gamma \left(K-1\right)^2}\right) = x^{\gamma + o(1)}$$
hence completing the proof. The reader can check that a similar calculation yields an upper bound. $\square$
Although any improvements on the density is still beyond our reach, some analogues of this idea have been suggested and they have been able to yield similar results in more general settings. They have some excellent potential of their own, but that is a topic for a different post!
Acknowledgement : I want to express my heartfelt thanks to my supervisor, Prof. Ramachandran Balasubramanian, who introduced me to the beautiful topic of Sidon sets. His straightforward approach to understanding mathematics has greatly influenced my own mathematical intuition and helped me greatly in understanding this paper.
