Introduction

Candidate gene-based association studies performed over many years have focused on various genes within the serotonergic system to identify susceptibility variants for alcohol dependence (AD). Serotonin mediates the reward effects of alcohol along with a network of other neurotransmitters. Previous serotonergic gene-based association studies in AD have been analyzed mostly for single-gene effects, especially in the serotonin transporter (5-HTT) gene (SLC6A4), that yielded modest effect sizes. Yet the serotonergic system is complex, consisting of numerous pre- and post-synaptic receptor subtypes in addition to the transporter system that regulates synaptic amounts of serotonin (Hayes and Greenshaw 2011). Among the serotonergic receptor subtypes in the human nervous system, 5-HT3 receptors are the only known ion-channel receptors evoking fast excitation of serotonergic neurons (Sugita et al. 1992). Acute exposure to alcohol potentiates 5-HT3 receptors at concentrations that produce intoxication (Narahashi et al. 2001; Sung et al. 2000), possibly through altered receptor affinity for serotonin (Narahashi et al. 2001), stabilization of open-channel state (Zhou et al. 1998), or increased release of serotonin (Lovinger 1997). Chronic heavy drinking depletes pre-synaptic serotonin reserves, causing a hypo-serotonergic state that may up-regulate post-synaptic 5-HT3 receptors (Johnson 2000).

The 5-HT3 receptors are assembled in homo- or hetero-pentamers formed by 5-HT3A and 5-HT3B subunits. The 5-HT3A subunit harbors the serotonin binding site (Hodge et al. 2004), and the 5-HT3B subunit is important for trafficking and stabilizing of the 5-HT3AB receptor complex at the cell membrane (Massoura et al. 2011). Two lines of evidence suggest a concerted effect of both subunits on serotonergic dysfunction underlying AD. The first is based on the location of the subunits within the human brain. Whilst the 5-HT3A homomers are uniformly located in various parts of the central and peripheral nervous systems, the 5-HT3AB heteromers are located predominantly in mesocorticolimbic structures implicated in alcohol and other drug addictions. The second is evidence from biochemical studies of greater conductivity with 5-HT3AB hetero-pentamers compared with 5-HT3A homo-pentamers (Lochner and Lummis 2010), whereas 5-HT3B by itself appears to be non-functional (Davies et al. 1999).

The genes encoding 5-HT3AB, namely, HTR3A and HTR3B (Davies et al. 1999), are located next to each other on chromosome 11q23.1 spanning a 90-kbp region (Miyake et al. 1995). So far, only two studies have examined the associations of HTR3A and HTR3B with AD (Ducci et al. 2009; Enoch et al. 2011). In a population of Finnish alcoholics who also had antisocial personality disorder (ASPD), Ducci et al. (2009) showed that the A allele of intronic HTR3B SNP rs3782025 was associated with a higher risk of AD + ASPD. A more recent study conducted in African-American AD men reported an association of the C allele of non-synonymous exonic HTR3B SNP rs1176744 with a higher risk for AD with a relatively large effect size for a complex trait (OR = 1.6; 95 % CI 2.1, 16.6) (Enoch et al. 2011). Additionally, in a pharmacogenetic trial based on an a priori hypothesis, Johnson et al. (2011) demonstrated the influence of functional variants of the serotonin transporter gene (SLC6A4) on conductivity via post-synaptic 5-HT3AB receptors. In that study, drinking severity was improved more by the 5-HT3 antagonist ondansetron in Caucasian alcoholics carrying two genotypes (5′-HTTLPR:LL and rs1042173:TT) in SLC6A4 that previously were characterized as associated with lower expression in alcoholics (Johnson et al. 2008; Seneviratne et al. 2009b).

In view of these molecular and genetic findings, we hypothesized that, in alcoholics, the combined effects of genetic variations regulating synaptic serotonin levels, coupled with greater expression and function of 5-HT3AB, may confer greater susceptibility to AD than they do individually. To test this hypothesis, in a population of AD and healthy individuals of European descent, we evaluated associations of HTR3A and HTR3B SNPs evenly covering the two genes and their interactions with the SLC6A4 5′-HTTLPR:LL and rs1042173:TT genotypes. Subsequently, we assessed the reproducibility of our findings in a much larger dataset from a genome-wide association study called OZ-ALC-GWAS.

Materials and methods

Primary analysis

Samples

The DNA samples used in the primary analysis were initially from 822 unrelated self-identified Caucasian subjects aged 18–65 years, which included 519 AD subjects and 303 controls. Three hundred and twenty-three of the AD individuals were enrolled at The University of Texas Health Science Center at San Antonio and the remainder at the University of Virginia. All AD individuals were currently drinking with no other DSM-IV axis I diagnosis (American Psychiatric Association 1994) other than nicotine dependence scored >8 on the Alcohol Use Disorders Identification Test (Bohn et al. 1995), and were part of two out-patient clinical trials that tested medications for alcoholism. The ethnicity- and age-matched control individuals were selected from a large genetic study on nicotine addiction and were recruited primarily from the Mid-South States in the US (Texas, Tennessee, Mississippi, and Arkansas) during 1999–2005 (Li et al. 2005; Seneviratne et al. 2009a; Sun et al. 2008). The control individuals had no history of substance abuse or other DSM-IV axis I diagnoses. At all sites that recruited case or control subjects, DSM-IV axis I disorders were diagnosed using the Structured Clinical Interview (First et al. 1994) for Diagnostic and Statistical Manual of Mental Disorders, 4th edition (American Psychiatric Association 1994) Axis I Disorders.

Percentage of females in the AD population was 30.4 and 64.0 % in the control population. The mean age (years ± SD) at study entry for males and females in the AD group was 43.87 ± 10.83 and 43.79 ± 12.81 in the control group. All participants provided written informed consent according to a protocol approved by all involved Institutional Review Boards.

Genotyping

Detailed information on all SNP locations, chromosomal positions, allelic variants, minor allele frequency, and primer/probe sequences is summarized in Supplementary Table 1. All SNPs were selected from the National Center for Biotechnology Information (NCBI) dbSNP database (http://www.ncbi.nlm.nih.gov/SNP/) on the basis of their location within the gene, high heterozygosity (minor allele frequency ≥0.05) for common variants, and biological significance for uniform coverage of the gene (Fig. 1 and Supplementary Table 1). The three low-frequency variants (i.e., rs33940208 in HTR3A and rs2276305 and rs17116138 in HTR3B) were examined, as they were located close to known common functional variants or in exonic regions.

Fig. 1
figure 1

Distribution of HTR3A and HTR3B SNPs and haplotype blocks. Less-frequent variant SNPs are indicated in lighter color font

Full size image

The DNA extractions and genotyping of all SNPs and the long (L) and short (S) alleles of 5′-HTTLPR were carried out as described in our previous publication (Seneviratne et al. 2009b). Briefly, SNP genotyping was performed with 50 ng of DNA amplified in a total volume of 10 μl containing 0.25 μl of MGB probe and TaqMan universal PCR master mix. Allelic discrimination analyses were performed on the ABI Prism 7900HT sequence detection system. To ensure the quality of genotyping, four no-template negative controls and four positive controls were added to each 384-well plate. The 5-HTTLPR L and S alleles were also genotyped with 50 ng of genomic DNA PCR amplified with the primers listed in Supplementary Table 1 and the PCR conditions as described in our earlier publication (Seneviratne et al. 2009b). The PCR products were electrophoresed on a 3.5 % agarose gel, and the amplicons were identified using ethidium bromide staining. About 5 % of the sample was genotyped in duplicate for additional quality control.

Assessment of sample admixture using ancestry-informative markers

The DNA samples from all 822 subjects were genotyped with 24 ancestry-informative markers (AIMs) to test for potential population stratification. These markers have high-frequency differences for South American/European ancestry and European/West African ancestry (Mao et al. 2007). For detailed information, please see our previous publications (Johnson et al. 2011; Seneviratne et al. 2009a). The program Structure (http://pritch.bsd.uchicago.edu/software/structure2_2.html) was run using the 24 AIM datasets to assess population substructure and to estimate genetic ancestry proportion scores for each participant. We analyzed the data set with K = 2 through K = 10, and the simulation parameters were set to 10,000 burn-ins and 10,000 Markov chain Monte Carlo iterations. The K value with the highest probability of capturing stratification was 3; hence, we obtained the ancestry proportion estimates by analyzing the dataset assuming three parental populations (K = 3) and the presence of population admixture. The average rate of missing genotype information for the 24 AIMs was less than 5 % for the entire dataset of 822 subjects. A small group of 42 individuals was identified with 0 % probability to belong to the main population cluster, and thus was excluded from all statistical association analyses described below. A summary of clustering results for the 780 subjects is given in Supplementary Figure 1. Individual genetic ancestry proportion scores for each participant were used as covariates in all association analysis models.

Statistical analysis

All variants except for rs1176746 within HTR3B conformed to the Hardy–Weinberg Equilibrium (HWE). Hence, rs1176746 was excluded from association analyses, and the remaining variants (10 HTR3A SNPs, 11 HTR3B SNPs, and 2 SLC6A4 variants) were tested for single- and multiple-locus associations with AD.

Single-locus association analysis with AD

Logistic regression and Chi-square tests were performed in SAS (v. 9.1) to analyze genotype and allelic associations of all 23 individual SNPs in HTR3A, HTR3B, and SLC6A4 with AD. For genotype association analysis, we tested additive, dominant, and recessive models, and all statistical models were covariate-adjusted with genetic ancestry scores, age at study entry, and sex.

Multiple-locus association and interaction analyses with AD

We examined the effects of multiple loci in each of the three genes on AD by assessing haplotype and gene–gene interaction analyses.

Linkage disequilibrium (LD) and haplotype analysis: The LD matrices and blocks were determined for all samples using Haploview (v. 4.1) under the analysis criteria of Solid Spine of LD > 0.8 (Barrett et al. 2005). The haplotypes located within each LD block were inferred, and their statistical associations with AD were analyzed using HaploStats (v. 1.2.1) (http://mayoresearch.mayo.edu/mayo/research/biostat/schaid.cfm) following two approaches. As the LD between common tag SNPs and rare or low-frequency variants is low (Liu and Leal 2012), our first approach was to analyze associations of AD with haplotypes located within each LD block formed only by common alleles, excluding the low-frequency alleles located in the midst of haplotypes formed by the common variants (see Fig. 1). The second approach was to include the low-frequency alleles within the LD block to assess their contribution to the haplotypes formed by the common variants. We defined a ‘major haplotype’ as one with a frequency of >5 % in either the AD or the control sample. Bonferroni correction was used to calculate the corrected P value for haplotype analyses for individual haplotypes within each block.

Interactive effects of genotypes at multiple loci in HTR3A, HTR3B, and SLC6A4 on AD: We assessed the interactive effects of genotypes located at multiple loci within and among HTR3A, HTR3B, and SLC6A4 on AD susceptibility using a two-step statistical analysis. In the first step, interactive variant combinations were identified employing a Generalized Multifactor Dimensionality Reduction (GMDR beta version 0.7) (Lou et al. 2007). The GMDR identifies interactive variant combinations with the highest impact on phenotypic variation (high vs. low risk) using a score statistic calculated by accounting for covariates (age, sex, and genetic ancestry). First, an exhaustive computational search was performed for all possible two- to five-locus variant combination models using all 20 common variants. Then, the GMDR selected the best variant combination model(s) for a given order (i.e., two- to five-order combinations) on the basis of their P value derived from the nonparametric sign test, cross-validation consistency (CVC), and testing balanced accuracy (TBA) (Lou et al. 2007). All two- to five-locus variant combinations with sign test P values of <0.05 and CVC of 0.6 were then tested to gain empirical P values for their prediction accuracy using 106 permutations. Following identification of the best variant combinations, specific genotype combinations within the above-detected statistically significant SNP combinations were analyzed for their effects on AD.

Replication analyses using OZ-ALC-GWAS dataset

A total of 6,699 samples representing 2,313 families from the OZ-ALC-GWAS study were included in the replication analysis. A detailed description of the study can be found elsewhere (dbGaP Study Accession: phs000181.v1.p1) (Knopik et al. 2004; Lynskey et al. 2005). Among the individuals who had genotype calls, 892 had missing phenotype information, but all 2,313 families had at least one individual with the required data.

Imputation of non-genotyped SNP data

A total of 370,404 SNPs were genotyped in the OZ-ALC-GWAS study using the Illumina genotyping platform “HumanCNV370v1.” Because genotype information was available for only 7 of the 23 SNPs genotyped in the primary sample, we performed imputations using IMPUTE v. 2.2 (Howie et al. 2009) to generate genotype calls for the following ungenotyped SNPs: rs1062613, rs33940208, rs1176722, rs1176720, rs1176719, rs10160548, rs1150220, rs1176713, rs3758987, rs4938056, rs12270070, rs2276305, rs17116138, rs17614942, and rs1042173. We used pilot CEU haplotypes from the 1000 Genomes project in a 0.59-MB region covering both HTR3A and HTR3B on chromosome 11 and a 0.55-MB region covering SLC6A4 on chromosome 17 (Genomic built 36) as reference panels for imputing HTR3A/B and SLC6A4 SNPs, respectively.

5′-HTTLPR L/S allele predictions

Because the replication sample came from a GWAS study where only SNPs were assayed, we predicted the 5′-HTTLPR 44 bp insertion/deletion (L/S) variant in SLC6A4 using a two-SNP haplotype proxy consisting of rs2129785 and rs11867581, as described by Vinkhuyzen et al. (2011). Briefly, we constructed haplotypes for each individual using a tabulated haplotype phase for each individual in PLINK (http://pngu.mgh.harvard.edu/purcell/plink/) (Purcell et al. 2007), using genotype data for SNPs rs2129785 and rs11867581 available in the OZ-Alc-GWAS data set. The haplotypes rs2129785:A–rs11867581:G and rs2129785:G–rs11867581:A were used as proxies for the 5′-HTTLPR L allele and haplotype rs2129785:A–rs11867581:A as proxy for the 5′-HTTLPR-S allele.

Interaction analysis

Following imputation tests, similar to the analyses in our primary sample, all 23 variants were analyzed for their single- and multiple-locus associations using software packages FBAT (Laird et al. 2000) and PLINK and for gene-by-gene interaction using the GMDR program as described above.

Results

Primary analysis

As mentioned in “Materials and methods”, the control sample had more female subjects than the AD sample, and for this reason, sex was included as a covariate in all analyses. Allele and genotype distributions for all HTR3A, HTR3B, and SLC6A4 variants in the control sample were similar to the frequencies reported in the HapMap data set (Supplementary Table 1). All HTR3A and SLC6A4 variants conformed to the HWE with the exception of SNP rs1176746 in HTR3B that significantly deviated from HWE (P = 9.0 × 10−4) in the control population, which thus was eliminated from subsequent association and interaction analysis. The overall genotype error rate was <1 %.

Single-locus analysis

The results from the allele and genotype association analyses for individual variants are presented in Table 1. Four common SNPs in HTR3B (i.e., rs3758987, rs4938056, rs3782025, and rs1672717) and the 5′-HTTLPR in SLC6A4 were marginally associated with AD. Of these, only the association of AD with rs3782025 in HTR3B remained significant after correcting for multiple comparisons with the SNPSpD program (Nyholt 2004). Among the three low-frequency SNPs examined, minor alleles of both rs33940208 in HTR3A and rs2276305 in HTR3B were significantly protective against AD with ORs of 0.212 (95 % CI 0.073, 0.616; P = 0.004) and of 0.261 (95 % CI 0.088, 0.777; P = 0.016), respectively.

Table 1 Significant association of genotypes and alleles with AD in single-locus analysis in primary population
Full size table

Haplotype-based association analysis

We detected two LD blocks within HTR3A and two LD blocks within HTR3B (Fig. 1). In HTR3A, neither the haplotypes formed by the 6 common SNPs within Haploview-defined “LD Block 3,” nor the haplotypes formed by the four common SNPs within LD Block 4, were significantly associated with AD. Within LD Block 3, the low-frequency SNP rs33940208 and the common SNP rs1062613 located 71 bp upstream of rs33940208 formed a two-SNP minor haplotype (rs1062613:C–rs33940208:T) with a nominally significant protective effect against AD (haplotype score = −2.2259; P value for individual haplotype association analysis = 0.0260 and global P = 0.0347; OR = 0.3706; 95 % CI 0.1283, 1.0705).

In HTR3B, we detected a 5-SNP haplotype in Haploview-defined “LD Block 2” with a significant inverse association with AD (defined as haplotype 1 in Table 2). The LD Block 2 is located in and around four exons that are common to all mRNA isoforms of HTR3B. As shown in Fig. 1, two low-frequency SNPs (rs2276305 and rs17116138) also reside within the region covered by LD block 2.

Table 2 Haplotypes within “LD Block 2” significantly associated with ad in primary population
Full size table

To further examine the effect of two low-frequency SNPs, we re-analyzed haplotype 1 with the addition of major alleles of rs2276305 and rs17116138 (defined as haplotype 2 in Table 2). The haplotype consisting of minor allele of rs17116138 together with the other six alleles of haplotype 2 (rs12270070:G–rs1176744:A–rs2276305:G–rs17116138:G–rs2276307:A–rs3782025:A–rs1672717:A) was not significantly associated with AD; and the haplotype consisting of the minor allele of rs2276305 had a very low frequency (0.96 %) in our primary sample (haplotype 3 in Table 2). Thus, we were not able to assess whether haplotype 3 would confer a greater protective effect than haplotype 2 or the individual rs2276305:A allele.

Gene–gene interaction analysis

As shown in Table 3, we detected two four-variant interaction models with significant genetic interaction effects on AD, with an empirical P < 10−6, cross-validation consistency (CVC) of 10, and test accuracies (TA) >50 % based on 106 permutation tests. Because three of the four variants included in the two interaction models were identical, we combined the two four-variant models, and the resulting five-variant model was assessed for interactions with AD (third model in Table 3). The five-variant interaction model had a significant genetic interaction effect on AD, albeit with a smaller P value than the two four-variant models, possibly because of the smaller sample size for the five-variant interaction model. Subsequent to detecting the interaction models, we analyzed the “genotype combinations” within the two four-variant and the five-variant interaction models (see right-side panel of Table 3). As shown clearly in Table 3, persons with all three genotype combinations had greater risk of AD than did those with individual constituent genotypes: rs3782025:AG (OR 1.657; 95 % CI 1.187, 2.314), 5′-HTTLPR:LL/LS (OR 1.564; 95 % CI 1.058, 2.311), rs1176744:AG (OR 1.325; 95 % CI 0.955, 1.838), and rs1042173:TT/TG (OR 1.052; 95 % CI 0.681, 1.624). Thus, our results provide evidence for strong genetic interactions within the serotonergic system underlying AD.

Table 3 Most significant SNP Combinations detected by GMDR program and best constituent genotype combination for gene × gene interactions in primary population
Full size table

Replication analysis in OZ-ALC-GWAS sample

The replication sample consisted of 1,739 DSM-IV-defined alcohol dependents, 4,068 control individuals, and 892 individuals without phenotypic data (affection status). Females were 37.72 % of the AD sample and 56.51 % of the control sample. The mean age (±SD) was 40.94 ± 10.75 in the ADs and 45.85 ± 8.29 in the control subjects. All statistically significant associations (P < 0.05) detected in the primary analyses were examined in the replication sample.

Imputation of missing SNPs and 5′-HTTLPR L/S allele predictions in OZ-ALC-GWAS study

Concordance rates for the 16 imputed SNPs using IMPUTE (v 2.2) ranged from 94 to 100 %, and proportions of missing genotypes were below 6 % for all imputations. The two proxy haplotypes for the 5′-HTTLPR:L allele, rs2129785:A–rs11867581:G, and rs2129785:G–rs11867581:A were present in the OZ-ALC-GWAS sample at frequencies of 0.436 and 0.110, respectively. The frequency of haplotype rs2129785:A–rs11867581:A, proxy for the 5′-HTTLPR:S allele, was 0.454. These estimated frequencies are similar to the L and S allele frequencies detected in the primary sample (0.546 vs. 0.550 for L and 0.454 vs. 0.451 for S).

Single-locus analysis

As shown in Table 4, two low-frequency alleles within HTR3A and HTR3B showed protective effects against AD in the OZ-ALC-GWAS sample: rs2276305:A (χ 2 = 7.224; P = 0.007) and rs33940208:T (χ 2 = 3.316; P = 0.0686). Genotype association analyses for rs2276305:AA/AG and rs33940208:TC/TT also revealed inverse associations with AD (Z = −2.555, P = 0.0106 for rs2276305:AA/AG and Z = −2.014, P = 0.0440 for rs33940208:TC/TT). None of the common variants that were significant (P = 0.05) at the single-locus level in the primary analyses was significant in the OZ-ALC-GWAS sample. However, the rs10160548:T allele, which was included in two interaction models presented in Table 3, was over-represented in ADs in the OZ-ALC-GWAS sample at the single-locus level (Z = 2.542; P = 0.0110).

Table 4 Significant individual genetic associations and gene × gene interactions for AD in replication population
Full size table

Haplotype analysis

The two major haplotypes within HTR3B LD block 2 associated with AD in the primary sample (haplotypes 1 and 2 in Table 2) were not associated with AD in the OZ-ALC-GWAS sample. However, the rare haplotype detected in the primary sample (haplotype 3 in Table 2) consisting of the low-frequency allele rs2276305:A was present in the OZ-ALC-GWAS with a large enough sample (16 families; frequency = 0.007) to perform statistical analysis. The haplotype rs12270070:G–rs1176744:A–rs2276305:A–rs17116138:G–rs2276307:A–rs3782025:A–rs1672717:A showed a protective effect against AD with a Z score of −2.596 (P = 0.009).

Gene–gene interaction analysis

All three genotype combinations found to have an interactive effect on AD in the primary sample also had significant interactive effects on AD in the OZ-ALC-GWAS sample, with a Z value of 3.155–4.384 and P value of 0.0016–0.00001 (see Table 4 for details).

Discussion

In the present study, we detected and replicated the finding on the influence of three genotype combinations on the risk for AD and that of two low-frequency alleles protective against AD. The five interacting risk genotypes within SLC6A4 (5′-HTTLPR:LL/LS; rs1042173:TT/TG), HTR3A (rs10160548:GT/TT), and HTR3B (rs1176744:AC; rs3782025:AG) are variants previously shown to be associated with alcoholism or related intermediate phenotypes. As hypothesized, these interacting genotypes were linked to about a threefold greater risk of AD than their constituent individual genotypes. Importantly, allele-based functional differences for three of these five variants (i.e., SLC6A4: 5′-HTTLPR:LL/LS and rs1042173:TT/TG; HTR3B: rs1176744:AC) were demonstrated to be functional in earlier molecular studies. More specifically, alcoholics carrying the 5′-HTTLPR:L allele have lower serotonin transporter binding and reuptake activity than homozygous 5′-HTTLPR:S carriers according to an earlier study conducted by our group (Johnson et al. 2008). Similarly, the rs1042173:T allele was associated with lower expression of serotonin transporters (Lim et al. 2006; Seneviratne et al. 2009b). Thus, the 5′-HTTLPR:L and rs1042173:T alleles, in combination, may lead to a synaptic hyperserotonergic state. The third known functional variant included in the risk genotype combinations was the HTR3B non-synonymous SNP rs1176744 (A/C), which results in a tyrosine to serine change at the 129th amino acid residue in the extracellular N-terminal domain of 5-HT3B (Walstab et al. 2008) close to the 5-HT3AB interface (Krzywkowski et al. 2008). This amino acid substitution significantly increases 5-HT3AB ion channel open time, augmenting serotonergic signaling (Krzywkowski et al. 2008). In a recent study, Enoch et al. (2011) reported a higher risk of AD associated with high activity of the serine allele in African-American male alcoholics. Further, several other studies reported associations of the rs1176744 serine allele with other psychiatric disorders that often are co-morbid with AD in Caucasians (Frank et al. 2004; Hammer et al. 2009, 2012; Yamada et al. 2006). It should be noted that, unlike the study by Enoch et al., in our study, it was the heterozygous genotype of rs1176744 that contributed to all three risk genotype combinations and also showed a trend to risk at the single-locus level (OR = 1.325; 95 % CI 0.955, 1.838; P = 0.092). This suggests a molecular heterosis effect that could be race specific. From a structural point of view, our findings can be interpreted as co-expression of two 5-HT3B subunits in the same individual: one with the amino acid serine at position 129 and the other with tyrosine (“heterozygous” 5-HT3B subunits) leading to a higher risk of AD. In such a situation, it is hard to decipher which variant is more prevalent or the change of direction of serotonergic singling associated with the genotype. However, using cultured tsA-201 cells without any treatment, Krzywkowski et al. (2008) demonstrated that expression of “heterozygous” 5-HT3B subunits in the pentameric 5-HT3AB receptor complex would mimic the heterozygous carrier state. Expression of “heterozygous” 5-HT3B subunits resulted in an increase in the time the channel was open compared with 5-HT3AB complexes expressing two homozygous wild-type 5-HT3B subunits with the tyrosine residue (Hammer et al. 2009; Krzywkowski et al. 2008). Longer open time of the 5-HT3AB channels can lead to enhanced responsiveness of neurons to serotonin. This enhanced responsiveness associated with rs1176744:AG may be heightened by the greater availability of synaptic serotonin that is associated with the SLC6A4: 5′-HTTLPR:LL/LS and rs1042173:TT/TG genotypes.

The molecular mechanisms underlying allelic differences of the other two SNPs included in our risk genotype combinations, rs10160548 in HTR3A and rs3782025 in HTR3B, are not yet elucidated. Nevertheless, as intronic SNPs, it is possible that both rs3782025 and rs10160548 alter mRNA expression through alternate splicing. Furthermore, Ducci et al. (2009) reported an association of the rs3782025:A allele with a higher risk of AD + ASPD in Caucasians. Perhaps because of the inclusion of alcoholics with and without ASPD, our findings indicate an association of AD only with the heterozygous genotype of the rs3782025:A allele.

The second main finding of our study was the protective effects against AD shown by the two low-frequency synonymous variants rs33940208 in HTR3A exon 1 and rs2276305 in HTR3B exon 5. One important caveat to these findings, however, is that their significance values did not reach the threshold for multiple statistical tests. Nevertheless, these associations had medium to large effect sizes in the primary analysis (Cohen’s d for allelic associations of rs2276305:A and rs33940208:T with AD 0.74 and 0.86, respectively), which were validated by replication in the OZ-ALC-GWAS sample. To our knowledge, low-frequency variants with a protective effect on AD have not been reported previously in European populations. As both rs33940208 and rs2276305 are located within 100 bp of two well-characterized functional common variants (see Fig. 1), they may modulate the functionality of the common variants. The rs33940208:T allele was seen only in carriers of rs1062613:C: The C allele of rs1062613 has been associated with lower HTR3A expression (Kapeller et al. 2008) and the CC genotype with high anxiety and greater amygdala responsiveness to emotional stimuli in humans (Iidaka et al. 2005; Kilpatrick et al. 2011). As high anxiety is a risk factor for AD, one would expect the protective allele to be linked to the non-risk T allele. On the other hand, it is tempting to speculate that the protective effects of the rs33940208:T allele might offset reductions in transcription associated with the rs1062613:C allele, constituting a sub-population with greater expression than in the carriers of the rs1062613:C–rs33940208:G haplotype. Similarly, the other protective allele detected in our study, rs2276305:A, was seen only in carriers of the major allele of the common functional SNP rs1176744:A, which previously has been reported as the non-risk allele (Enoch et al. 2011). Taken together, these findings support the view that functionality attributed to common variants is in fact a consequence of “synthetic associations” arising from combined effects of multiple rare variants in their vicinity (Goldstein 2009). In fact, several examples of similar rare-common SNP modulating effects on other common disorders with high heritability rates such as AD have been reported (Raychaudhuri 2011). Thus, deep sequencing of HTR3A exon 1 and HTR3B exon 5 regions might help detect multi-variant modulating effects on the common functional SNPs rs1062613 and rs1176744.

The present study has two unique strengths. First, we analyzed an ethnically homogenous sample selected using self-reports of ethnicity and then subjected the sample to an additional “filtering” step based on their genetic ancestry using an ancestral marker panel. Second, we strengthened the reliability of our findings by replicating them in a larger independent family-based sample. Both the primary and the replication data set consisted of heavy drinkers who met the criteria for DSM-IV alcohol dependence, and the sex, age, and ethnic compositions were similar. Thus, we believe the two data sets were phenotypically similar. Also, the family-based structure of the replication data set provided us with enhanced statistical power, as risk or protective variants for common diseases segregate over several generations (Zhu and Xiong 2012). In addition to the strengths in the study design, a parallel study conducted recently by our group revealed that rs10160548 in HTR3A, rs1176744 in HTR3B, and 5-HTTLPR and rs1042173 in SLC6A4, which conferred risk for AD, also interacted significantly with each other to influence the risk of nicotine dependence in both Caucasians and African Americans (Yang et al. 2013). A minor concern about the present study is that most of the variants analyzed in the replication dataset were imputed or assessed by employing statistical algorithms rather than direct genotyping of the DNA samples. However, as the reliability rates were above 95 % for all imputations, we do not believe our findings would be affected by this fact.

In conclusion, our findings provide strong evidence for genetic variability underlying AD risk conferred by the interactive effects of SLC6A4, HTR3A, and HTR3B and low-frequency variants, especially some located close to common functional variants. Next-generation sequencing technologies may enable further refining of the identification of genetic variability within SLC6A4, HTR3A, and HTR3B that contributes to the risk of AD and related phenotypes.