Introduction

Autism is a highly heritable but genetically complex neurodevelopmental disorder with an onset early in childhood. It is characterized by significant disturbances in language and reciprocal social interactions, combined with repetitive and stereotypic behaviors. Autism is not a distinct categorical disorder but instead represents one extreme of a spectrum of social and communication impairment and behavioral problems, referred to as autism spectrum disorders (ASD). Prevalence estimates for autism have substantially increased in the last decade. The incidence of severe autism is estimated as one in 1,000 individuals, with males affected at a rate four times that of females [1, 2]. The incidence increases to approximately two to three in 1,000 when the broad diagnosis is considered, including milder forms of ASD [3, 2]. Evidence from various studies indicates that ASD has a strong but genetically complex etiology, possibly involving epistasis and locus heterogeneity. While rare single mutations or chromosomal abnormalities are likely responsible for some cases, current models strongly suggest that inheritance of multiple interacting polymorphic loci contributes to a continuum of disease phenotypes in the majority of affected children [4, 5]. Twin and family studies support a multilocus etiology with as many as five to 20 loci being involved in its manifestation. Twin studies show a concordance of 60% among monozygotic (MZ) twins and 0% among dizygotic (DZ) pairs for classic autism but increases to 92% for MZ and 10% for DZ pairs when the broader phenotype of related social and language abnormalities are included [6–8]. Interestingly, the milder phenotypes are similarly elevated in other relatives of singleton probands, supporting the hypothesis that autism phenotypes are expressed across a spectrum of severity [9]. The recurrence risk for siblings of autistic probands is approximately 3% for classic autism and up to 10% when all disorders within the spectrum are considered. While the observed sibling recurrence risk is low, the relative risk compared to the general population is 50–100 times that in the general population [7, 8, 10]. Heritability is therefore estimated at 90%, which is among the highest for psychiatric disorders.

The high heritability of autism has driven the efforts of numerous groups to search for susceptibility loci using genome-wide linkage screens in multiplex families [11–18] and genome-wide association studies [19]. Although this genomic approach has yielded multiple suggestive regions, a specific risk locus has yet to be identified. Examination of families with only affected males has been suggested [20–22] as a method of looking at a more homogeneous autism subset. In addition to looking at the male-only subset, we have taken the alternative approach of selecting a set of functional candidate genes that enables us to examine each gene in more detail. This approach entails selecting genes based on knowledge of the specific phenotypes and our knowledge of the underlying neurobiology related to those behavioral abnormalities in individuals with autism. The extensive variation in phenotypes and severities within ASD suggests the involvement of multiple predisposing factors, interacting in a complex manner with normal neurodevelopment channels and pathways. The biological evidence supporting the involvement of neurobiological pathways, in particular the serotonin pathway (Fig. 1), is compelling.

Fig. 1
figure 1

Representation of the serotonin pathway with the metabolic genes indicated. The other genes genotyped in this study are YWHAZ TPH’s activation protein. MTNR1A and MTNR1B melatonergic receptors, HTR3A and HTR2A serotonergic receptors, SLC7A5 tryptophan transporter, SLC6A4 serotonin transporter

Full size image

The serotonin pathway was first implicated in 1961 when an elevation of whole blood serotonin (5-hydroxytryptamine, 5-HT) levels in patients with autism was reported [23]. Since the initial report, a number of replication studies have shown hyperserotonemia in 25% to 50% of patients with autism [24, 25]. Other studies have shown that more than 99% of whole blood serotonin is contained in platelets [26, 27] and that platelet serotonin accounts for the hyperserotonemia found in autism [27]. Thus, elevated serotonin levels are perhaps the most consistent pathophysiological finding in autism. Furthermore, deficits in the levels of tryptophan (a precursor in 5-HT synthesis) in adults with ASD increase the stereotypic behaviors associated with autism [28] and data from functional imaging studies indicate altered serotonin synthesis rates in children with autism versus nonautistic controls [29]. Additional affirmation for the involvement of the serotonin system in autism is the positive response individuals display when treated with selective serotonin reuptake inhibitors for disruptive and aggressive behavior [30–32]. Several recent gene association studies have attempted to identify gene variants important for serotonin system function to show that the variant of interest is linked to autism susceptibility. The studies, mostly focused on the serotonin transporter locus, SLC6A4, provide marginal evidence that genetic variation can impact the level of expression of the transporter [33] and that multiple and rare allelic variant might increase the risk of susceptibility to autism [34]. However, even though variation in SLC6A4 was associated to autism in some studies [35, 34], in other studies, it showed no evidence for association (summarized in [36]). The results, when viewed collectively, provide solid but inconsistent evidence that dysfunctional serotonin signaling plays an integral role in the development of autistic behaviors.

Materials and methods

Dataset

Our analysis was conducted on a dataset consisting of 403 Caucasian American families collected in the Southeast United States by the Center for Human Genetics Research at Vanderbilt University and the Institute for Human Genomics at the University of Miami (Table 1). The dataset is a mixture of multiplex and parent–child trio families with additional unaffected sibs in some cases (Table 1). Probands for the study consisted of individuals between the ages of 3 and 21 years who were clinically diagnosed with autism using Diagnostic and Statistical Manual IV (DSM-IV) criteria. The clinical diagnosis of autism was confirmed based on clinical evaluation using DSM-IV diagnostic criteria supported by the Autism Diagnostic Interview—Revised and medical records. Exclusion criteria included developmental level below 18 months, severe sensory problems (e.g., visual impairment or hearing loss), significant motor impairments (e.g., failure to sit by 12 months or walk by 24 months), or identified metabolic, genetic, or progressive neurological disorders. Parents/caregivers were informed of the purposes, risks, and benefits of participating in this project and provided informed consent.

Table 1 Dataset distributions
Full size table

Molecular analysis

Genomic DNA was extracted from blood using standard protocols and a commercial system (Puregene; Gentra Systems, Minneapolis, MN, USA). All single nucleotide polymorphisms (SNPs) were identified using the Ensembl (http://www.ensembl.org), dbSNP (http://www.ncbi.nlm.nih.gov/projects/SNP), and AppliedBiosystems (http://www.appliedbiosystems.com/) databases. Genes were selected based on their involvement in the serotonin pathway, including those that play a critical role in the synthesis and metabolism of serotonin, its transport within and across synapses and genes that affect serotonin receptor function. Multiple SNPs spanning each gene were chosen using a hierarchy of nonsynonymous coding change, minor allele frequency >0.10, and location within the gene. We calculated the linkage disequilibrium (LD) between SNPs in this dataset to assess the coverage of the genes with the goal of capturing most of the common variations (supplementary material). A total of 45 SNPs were genotyped for the ten genes (Table 2).

Table 2 Tested genes and single nucleotide polymorphisms
Full size table

SNPs were genotyped using the ABI 7900 Taqman system [37]. Laboratory personnel were blinded to pedigree structure, affection status, and location of quality control samples. Duplicate quality control samples were placed both within and across 384-well plates, and equivalent genotypes were required for all quality control samples to ensure accurate genotyping. Mendelian inconsistencies were identified using PedCheck [38]. Suspect genotypes were reread or retested.

Statistical analysis

Genotyping efficiency, Hardy–Weinberg equilibrium, and LD were checked using Haploview [39] and the genotypes generated in our study. Linkage analysis was conducted using two-point heterogeneity logarithm of odds scores (HLOD) calculated using FASTLINK and HOMOG [40]. Both recessive and dominant models with disease allele frequencies of 0.01 and 0.001, respectively, were analyzed. This approach is robust for detecting linkage signals when the underlying model is unknown or complex [41]. The pedigree disequilibrium test (PDT) assessed family-based allelic association and the genotype-PDT (GenoPDT) tested genotypic association to the risk of autism [42]. Taking into account the 4:1 ratio of males to females affected with autism, the HLOD, PDT, and Geno-PDT were also run in a subset of families containing only affected males (male only, N = 303). All statistical results are reported as nominal p values, except where specified. In the latter case, we performed a correction according to the method of Nyholt [43], which corrects for the effective number of independent tests taking the LD between SNPs into account.

Multifactor dimensionality reduction (MDR) analysis was used to detect multilocus interactions [44]. Since MDR is designed for case–control data, we extracted from any family with a complete parent–child trio (one per family for multiplex families) the genotype of the affected child. We constructed the genotype of the “pseudo” controls using the nontransmitted alleles of the parents [45]. We tested for all two-way and three-way interactions. MDR is a model-free method of analysis that generates empiric p values based on randomly permuting the data for all possible combinations, thus correcting for multiple comparisons. MDR has >80% power to detect both main and interactive effects in a dataset of this size, even in the presence of locus or genetic heterogeneity [46].

Results

No LOD scores >1.0 were observed in the overall dataset. In the male-only subset, we observed only a single LOD score >1.0 (1.36) for rs34532313 in MTNR1A (chromosome 4) in a dominant model (supplementary material).

We initially tested 75 SNPs in a subset of 241 families. Of these, 26 SNPs generated no LOD scores >1.0 or association p values <0.05 and four were later found not to be relevant to the pathway. Thus, a total of 30 SNPs were dropped from further consideration. We tested 45 SNPs in ten different genes in the full dataset of 403 families for association with autism risk. Nominally, significant results were obtained for SNPs in five different genes (YWHAZ, MTNR1B, HTR3A, SLC7A5, AANAT; Table 3). Our most significant p value was 0.0002 at rs1150220 in HTR3A, a serotonin receptor located on chromosome 11, the only result that survives a Nyholt correction (p = 0.022). The odds ratios for the heterozygote and homozygote are 1.46 (1.044, 2.042) and 1.251 (0.539, 2.090), respectively. Examination of the male-only dataset identified additional nominally significant results in MTNR1B (p = 0.006), a melatonin receptor on chromosome 11, SLC7A5 (p = 0.003) on chromosome 16, and a marginal association for AANAT (p = 0.01) on chromosome 17.

Table 3 Pedigree disequilibrium test
Full size table

MDR analysis for gene–gene interactions was performed across all 45 SNPs. A modestly significant interaction (p = 0.01) between rs10830962, near MTNR1B, and rs1007631, near SLC7A5 (chromosome 16), was observed (prediction accuracy = 59.6%; Table 4).

Table 4 Overall dataset MDR analysis results
Full size table

Discussion

Abnormalities in serotonin metabolism are one of the few consistent biological findings observed in autistic individuals. Although genome-wide linkage scans and candidate gene studies have implicated various loci in the serotonin pathway, the results have been inconsistent. Our study was designed to test two underlying hypotheses. First, does any prominent candidate gene in the serotonin system display a major locus effect in a previously unexamined dataset? Our data suggests that a single locus effect resides within HTR3A, which provided strong association results with a p value of 0.0002 (p = 0.022 corrected) at rs1150220. This result was not restricted to the male subset. This particular SNP is intronic, and there is no current data to suggest a functional consequence of this variation.

Second, do the serotonin system genes provide interesting multilocus interaction effects? This question is driven, in part, by the possibility of genetic buffering, where the moderate dysfunction of one or more genes (perhaps due to common variation) is compensated by other genes. Should more than one gene have moderately altered function, the entire system may then be perturbed. Our MDR analysis identified a moderate interactive effect between rs10830962 and rs1007631 (p value of 0.01) in a two-way interaction model in the overall dataset. Neither of these SNPs resides within the coding sequence of a gene, with rs10830962 near MTNR1B and rs1007631 near SLC7A5. There is no obvious biological explanation for this interaction, since the genes are not adjacent in the pathway and there is no evidence of direct biological interaction. However, it is possible that these SNPs, or others in LD with them, may have as yet unknown regulatory functions.

HTR3A is an interesting candidate gene. The product of HTR3A belongs to the ligand-gated ion channel receptor superfamily; it is permeable to Na+, K+, and C2+ ions. HTR3A is composed of four hydrophobic transmembrane segments, a large extracellular domain containing a Cys–Cys loop, a long intracellular segment between the third and the fourth transmembrane regions and an extracellular C terminus [47]. HTR3A is located on chromosome 11q23.1–23.2 and encodes subunit A of the type 3 receptor for serotonin and functions as a neurotransmitter, a hormone, and a mitogen. This receptor causes fast depolarizing responses in neurons after activation. It functions to mediate rapid synaptic transmission in the brain [48]. At the presynaptic location of the receptor, it mediates neurotransmitter release [49]. The gene consists of nine exons and spans approximately 15 kb [50]. HTR3A contains nine known coding variations (three nonsynonymous). HTR3A is highly expressed in the central nervous system as well as in the colon, intestine, and stomach [51]. For HTR3A to have the highest activity, it forms heteromeric combinations with HTR3B. However, HTR3A is unusual in that it can form homomeric assemblies with reduced channel activity [52]. Alternatively spliced transcript variants encoding different isoforms have also been identified.

The serotonin system has been widely investigated in neuropsychiatric disorders and has been implicated as having a role in learning, mood, thermoregulation, sleep, sexuality, and appetite [53–56]. Variations in HTR3A has specifically been linked to several mental disorders such as bipolar disorder [57, 58], harm avoidance in women [59], and schizophrenia [60]. Furthermore, HTR3B was associated to depression in a sample of Japanese women with major depression [61]. Krzywkowski et al. demonstrated that naturally occurring variation in the receptor created a drastic change in their function and expression [62]. Thus, dysfunction of HTR3A resulting in altered neurotransmission within the pathway is a possible source of insight in elucidating the etiology of the disorder.

Even though we captured a significant number of the common variation for these genes, our study is not fully comprehensive. There could be variation in regulatory elements for these genes outside of the coding regions, within the intronic regions not completely covered by our analysis and even outside the flanking regions selected in our study. There could be an extreme level of locus heterogeneity, which has a significant negative impact on power both for PDT and MDR analyses. Since we only examined common variation, the underlying effects could arise from multiple rare variants in one or more of these genes.