Department of Statistics Seminars

*** Please note this seminar is at 2pm. ***

A hidden Markov model includes as one component an underlying Markov chain whose states are assumed unobserved. A "not-so-hidden" Markov model has the same probabilistic structure, but the states of the Markov chain are either fully, or at least partially, observed. The experience in applying not-so-hidden Markov models in the environmental and geophysical sciences is reviewed. It is argued that this experience should provide some hints about the circumstances in which the use of hidden Markov models would be beneficial in environmental data analysis.

One example of a not-so-hidden Markov model is a chain-dependent process, a model commonly fitted to time series of daily weather (sometimes termed a "weather generator"). This model involves a stochastic process (e.g., precipitation intensity, temperature) defined conditional on the state of an observed Markov chain (e.g., precipitation occurrence). In its most general form, the temporal dependence in the observed variable is directly modeled, not simply induced from the hidden Markov component.

Much debate in the environmental and geophysical sciences revolves around the question of whether "regimes" exist. Conceptual models, with close resemblance to hidden Markov models, have been proposed as probabilistic representations of such regimes. A particular variant of a hidden Markov model, used to quantify climate predictability as consistent with regimes, involves a Markov chain whose states are observed, but whose transition probabilities shift depending on a hidden state. Examples of geophysical variables for which regime changes would have important environmental implications are numerous. They include indices of large-scale atmospheric/oceanic circulation, such as the North Atlantic Oscillation, as well as palaeoclimate reconstructions (i.e., evidence for "abrupt" climate change).

The role of hidden Markov models in environmental and geophysical data analysis is not yet clear. Besides serving as an empirical device for more flexible modeling, attempts have been made to attach physical interpretation to hidden state variables. So far, such interpretations seemed to have verified what is already known, rather than resulted in new discoveries.

• The National Center for Atmospheric Research is sponsored by the National Science Foundation.

To pull a dataset together for a graduate statistics class in 2005 I read the tracks from several of my CDs into a music editing program. I snipped out the first 40 seconds of each track, and converted this to numeric data using the R package, tuneR. I calculated descriptive statistics for each track, yielding 72 variables for 62 music clips, allowing us to study these questions:

Can I get my computer to recognize Rock from Classical tracks?

How do New Wave clips compare to Rock and Classical?

What are the similarities between the music from Abba, Beatles, Eels, Vivaldi, Beethoven, Mozart, and Enya?

In this talk we'll look at supervised classification into Rock and Classical classes, and cluster analysis of the music clips. One particularly interesting part of the cluster analysis is to look at self-organizing maps wrapping through high-dimensional data.

The Grammar of Graphics (Lee Wilkinson, 2005) provides a framework to allow us to move from describing plots with specific names (eg. pie chart, scatterplot, log-log plot) to describing a plot by the components that it is composed of. This makes it easy to describe a wide range of plots using a relatively simple series of components. These components include graphical objects, aesthetic mappings from data to visual properties, statistical transformations and coordinate systems.

I will introduce these ideas using examples produced by ggplot, an R package which implements the Grammar of Graphics. I will also discuss some of the differences between my implementation and the original grammar of graphics, including some issues related to moving from a declarative to a functional language.

http://had.co.nz/ggplot/2006-auckland.pdf

We present data, collected using baited underwater video, on the spatial distribution and population structure of the snapper /Pagrus auratus/ along the northeastern coast of Great Barrier Island. A variety of environmental variables are considered that may contribute to the patterns observed. The current distribution of snapper populations correlates with wave exposure. This pattern maybe confounded by spatial variation in fishing pressure.

A highly protected marine reserve has been proposed at a location on the northeastern coast of this offshore island. This reserve will provide the opportunity to control for the effects of fishing and allow further investigation of environmental variables influencing the distribution of snapper. The proposed reserve will have an approximate area of 500 km^2. We made qualitative and quantitative predictions of how snapper populations will respond to the release of fishing pressure if the reserve is created. We used an individual-based model constructed using data from existing coastal reserves. These coastal reserves are all smaller than 6 km^2. We present a range of predictions on how populations will respond to the release of fishing pressure and suggest further studies to investigate the role of environmental variables on snapper distribution.

A screening experiment is typically used at the start of a sequential investigation of a process. The idea is to run a small experiment to sift through a set of candidate factors and identify those that impact the response -- these are called the active factors and will be studied in detail in later experiments. A good screening design is one that can look at a large number of candidate factors in a limited number of runs and reliably identify those that are active. Recently, a number of "supersaturated" designs have been proposed which have greatly increased the number of factors that can be considered using a given number of runs. However, there are questions about the reliability of these designs.

This talk will explore the use of supersaturated designs for screening experiments and present a criterion for evaluating the ability of these designs to correctly identify the active factors. This talk is based on joint work with Professor Randy Sitter of Simon Fraser University.

Key problems for models of disease spread relate to threshold, velocity of spread, final size and control. All these aspects depend crucially on the network structure of individual interactions.

Networks of interest range from the highly localised case, where interactions are only between near neighbours, to the opposite global extreme where all interact equally with all, so that a disease can spread much more quickly through the population. Understandably, there has been much recent interest in `small-world' and meta-population models, in which a relatively small number of long-distance connections can change a network from local to effectively global. Such models seem particularly relevant to the changed patterns of human and animal diseases in a world whose connectivity, in terms of both travel and trade, has increased hugely in recent decades.

In consequence, a number of different mathematical and statistical approaches have been developed recently that focus on networks. I shall discuss the strengths and weaknesses of some of these approaches, with examples drawn from both human and animal diseases, susch as SARS, Foot and Mouth disease and avian flu. I shall also discuss the wider implications, as illustrating what mathematics can and cannot do in helping us predict and control disease outbreaks.

http://www.ma.hw.ac.uk/~denis/

DNA evidence recovered from a scene or collected in relation to in a case is generally declared as a mixture when more than two alleles are observed at several loci. However, in principle, all DNA profiles may be considered to be possible mixtures, even those that show no more than two alleles at any locus. When using a likelihood ratio approach to the interpretation of mixed DNA profiles it is necessary to postulate the number of potential contributors. However, this number is never known. In this talk we explore the risk associated with current practice, investigate the behaviour of current models, and examine possible ways to surmount this ambiguity.

This is joint work with John Buckleton from ESR and Peter Gill from the Forensic Science Service.

In clinical trials data is rarely perfectly observed and failure to account for missing data will at best lead to inefficiency.

Frequently data is assumed to be missing at random (MAR) or missing completely at random (MCAR), a stronger assumption. MAR assumes that the probability that an outcome is missing depends at most upon the observed data. Under MAR individuals with missing outcomes can be ignored and imputation methods may be used We consider a relaxation of MAR, latent ignorable (LI); if the baseline were perfectly observed then the outcome would be MAR.

We demonstrate the performance of a series of imputation methods and discuss where they may be inappropriate. We then introduce a Bayesian model framework for LI data. Within this framework a full sensitivity analyses can be carried out using expert knowledge. In particular we may consider that the missingness mechanism may be informative, where the probability of an outcome being missing depends on the value.

This talk will discuss the use of systematic samples for spatially autocorrelated ecological abundance data. I will show how Krige's Additivity Relationship (KAR) and geostatistics (variography) can be used to estimate the standard error of a (spatial) systematic sample. Computer simulation will compare this novel method with the most common systematic sample estimators.

Refreshments will be in the Statistics tea room on the 3rd floor.

Please refer to this PDF for abstract: Estimation and Testing with Interval-Censored Data.

The Fisher scoring and Gauss-Newton methods are two popular optimisation methods for well-formed statistical estimation problems. In this talk, I will show how both methods can be extended and used to provide fast solutions to complex optimisation problems which are previously considered difficult, for example, when the expected Fisher information matrix does not have a closed-form expression, or when there exist constraints on parameter values. Several applications will be given, including maximum likelihood estimation of finite mixtures/nonparametric mixing distributions, isotonic regression, and robust estimation.

I was recently involved in a project bringing together three quite disparate areas of the mathematical sciences: statistics; algebraic geometry; and computational biology. The resulting book (Pachter and Sturmfels, /Algebraic Statistics and Computational Biology/) does not sit easily with conventional divisions between pure mathematics, applied mathematics and statistics. In this talk I will introduce aspects of applied algebraic statistics, and then do some hand waving about how algebraic statistics is proving useful for reconstructing evolutionary history.

Introductory statistics papers generally have a large clientele of students who are majoring in other subjects. In these talk, we look at the reasons why multiple choice assessment is an appealing alternative to the use of traditional assessment methods, which are demanding of teacher time. We present our findings when tests comprised of multiple choice and free response questions are used as an assessment device in a large introductory statistics paper. The results indicate that multiple choice questions and free response questions are measurements of different attributes. This cautions us against over reliance on the use of multiple choice questions for assessment.

(Coauthored by William. M. Bolstad.)

The Marine Environment Classification (MEC) is a quantitative classification of New Zealand's exclusive economic zone that was build using a number of remotely-sensed environmental variables. The MEC is intended as a tool for conservation and fisheries management, and its utility in this context depends on how well it serves as a surrogate for broad-scale biological patterns. Using a variety of multivariate statistical methods, this study evaluated the performance of the MEC at discriminating important variation in a large observational dataset of shallow coastal rocky reef fish communities.

Margrabe provides a pricing formula for an exchange option where the distributions of both stock prices are log-normal with correlated Wiener components. Merton has provided a formula for the price of a European call option on a single stock where the stock price process contains a continuous Poisson jump component, in addition to a continuous log-normally distributed component. We use Merton's analysis to extend Margrabe's results to the case of exchange options where both stock price processes also contain compound Poisson jump components. A Radon-Nikodym derivative process that induces the change of measure from the market measure to an equivalent martingale measure is introduced. The choice of parameters in the Radon-Nikodym derivative allows us to price the option under different financial-economic scenarios. In the case of the American version of the exchange option, we decompose its price into a sum of the European price and an early exercise premium. A probabilistic intepretation for the early exercise premium is provided.

Growing elephant populations beleaguer many areas in Africa - they pester people and destroy other species. There must be too many of them! Not so says CITES, which only allows limited trade in ivory because elephants may go extinct. Ignoring the reason, much of elephant debate concludes that they must be managed by controlling their numbers.

The growing elephant numbers in southern Africa induced control methods such as the use of contraceptives, culling and translocations. These solutions assume that elephant damage is high when there are lots of them. I will illustrate that such an assumption treats symptoms of elephant effects and not the causes thereof. Treating causes follows a model of de-fragmenting landscapes which induces a space-time interface between elephants. It creates variance in density at a local scale that gives temporal respite from elephant pressure to other species as well as time for these to recover.

The spatial-impact model assumes that elephant populations are regulated. I show that elephant numbers display a full spectrum of decreases, increases and stability when there is no artificial water or the area is not confined. Density affects the growth of those that are stable. I then introduce a method to get birth and death rates. Birth rates decrease when densities increase - the mechanism of how density controls growth. I also show that droughts affect survival rates. The interplay between births, affected by density, and deaths, affected by droughts, leads to temporal variation in numbers.

However, such variation in numbers must be modulated by movements between sub-populations. I introduce a movement model and show how landscapes influence elephant movements. I then create a spatial model that includes both space and time to demonstrate how local instability in numbers, may lead to regional stability in a population. Such meta-population dynamics should reduce elephant effects on humans and other species on a local scale without compromising elephants on a regional basis.

Principal components analysis (PCA) relies on the eigenvalue/eigenvector decomposition. This produces vectors of loadings which have desirable properties such as orthogonality and uncorrelatedness. However the exactness with which these loadings vectors are calculated means that simplicity is compromised.

Using a natural definition of simplicity, namely proportionality to vectors of (hopefully low magnitude) integers, I will discuss an approach that focuses on the production of simple loadings vectors (Vines (2000)). I will show that simple loadings vectors can be produced without the need to resort to subjective rounding of loadings vectors and with little sacrifice of the optimal properties of eigenvectors.

The Jacobi-like nature of the algorithm means that variances are generally computed only along 1-dimensional projections. Thus, by the use of a 1-dimensional robust variance estimator, I will also show that one extension of the algorithm is to robust principal components analysis and hence the robust estimation of covariance matrices.

The Bayesian framework offers many possibilities for flexible and hierarchical modeling of data collected in case-control studies, but Bayesian athways have traditionally remained relatively less explored in this context. In this talk, I will first provide an overview of the current state of art in Bayesian analysis of case-control data and then present an application of Bayesian ideas to studies of gene-environment interaction.

In case-control studies of gene-environment association with disease, when genetic and environmental exposures can be assumed to be independent in the underlying population, one may exploit the independence in order to derive more efficient estimation techniques than the traditional logistic regression analysis (Chatterjee and Carroll, Biometrika, 2005). However, covariates that stratify the population, such as age, ethnicity and alike, could potentially lead to sources of non-independence. We provide a novel semiparametric Bayesian approach to model stratification effects under the assumption of gene-environment independence in the control population, conditional on stratification effects. We illustrate the methods by applying them to data from a population-based case-control study on ovarian cancer conducted in Israel. A simulation study is carried out to compare our method with other popular choices. The results reflect that the semiparametric Bayesian model allows incorporation of key scientific evidence in the form of a prior and offers a flexible, robust alternative when standard parametric model assumptions do not hold.

http://www.stat.ufl.edu/~mukherjee/

A company selling electricity into a market (as in New Zealand) must choose what offer to make so as to maximize its expected profit, allowing for stochasticity in demand and the behaviour of competitors. I will describe a method for solving this problem.

Statistics New Zealand is the National Statistical Office for New Zealand. Also it is the leader of the Official Statistics System (OSS), where the OSS encompasses all the statistical outputs of all areas of government. Aspects of Statistics New Zealand's OSS leadership include developing and promoting the use of protocols for data collections, the professional statistician's network and creating the Official Statistics Research Data Archive Centre (OSRDAC). OSRDAC is intended to be a central archive for OSS data that helps enable research of benefit to New Zealand to be done. Given the potentially large amount of data being collected and being placed in OSRDAC, access to that data becomes an even more important issue. Balancing the need for access to the data while preserving the confidentiality and privacy of the people and businesses that have provided that data is becoming a more involved and technically challenging issue. While I will cover these topics in the seminar I am happy at the end of it to take any questions raised about the OSS in general and try to arrange for someone in the OSS to answer them for you.

The Learning Centre had its origin in the early nineties when a change in the Victorian Higher School Certificate mathematics subjects resulted in a decline in skills and a large grant for "Special Assistance" allowed the Department to provide additional student support and redevelopment of courses at first year level. By the late nineties we had become the First Year Learning Centre with eight permanent Lecturer A's, a home area and a large contingent of sessional tutors. With further growth, in 2004 we became the Mathematics and Statistics Learning Centre and now aim to provide appropriate support for student learning in mathematics and statistics at all undergraduate levels as an integral part of the teaching/learning nexus for students in the Department.

Recent advances in the understanding of chaotic and complex systems are paradoxically pointing analysts to the use of simpler models to describe them. Such models, like the famous Game of Life, work because the complexity of the system comes from the multiple interaction of the parts, and the geometry of their spatial distributions, rather than the detail of the parts themselves. Thus models with simple representations of the components are often far more effective at generating simulations that are statistically similar to the real world. This talk discusses how relatively simple agent models can be effective tools for modeling a variety of complex systems, in particular, warfare and sports.

Spreadsheets, particularly Microsoft Excel, have become an indispensable tool for managers. Excel is widely used, and managers are comfortable in using it and are familiar with many of the facilities and features that it offers. Furthermore, many students are already familiar with spreadsheets before they start their statistics course and we should embrace and build on this expertise and acceptance. While specialist packages such as Minitab, SPSS, SAS and R are undoubtedly necessary for more advanced statistical work, we would strongly advocate the use of Excel for basic statistical computation, particularly on introductory courses and both under-graduate and postgraduate courses in business statistics.

Another useful feature of spreadsheets is that, with a little ingenuity, they can be used very effectively to demonstrate many statistical concepts. As such they provide a powerful teaching aid. About 10 demonstrations have been developed for use at various stages of an introductory course in Statistics. Most take no more than 5 or 10 minutes to run and can create an interesting diversion in the middle of a class, as well as reinforcing a key point. They have been developed in Excel, usually making use of macros and controls such as buttons and sliders, to create a dynamic demonstration. As well as discussing general issues in the use of Excel in Statistics, the talk will describe some of these demonstrations and discuss the associated learning points.

Gaussian Markov random fields (GMRF) are rife in spatial statistics, being convenient from both computational and theoretical viewpoints. Not uncommonly the GMRF is defined on a state space with dimension 10⁶ to 10⁹ in which case general sampling algorithms can be very slow. Efficient GMRF samplers exploit sparseness of the precision matrix within a Cholesky factorization to allow efficient sampling from the full GMRF, as well as marginal and conditional distributions. In this talk I present an alternative, sequential, algorithm derived from the conjugate gradient (CG) optimization algorithm. CG has the remarkable property of minimizing a quadratic form exactly in a finite number of steps while requiring storage of only two state vectors. The conjugate direction sampler has the analagous property of independence between output samples with more than a finite spacing (bounded above by the dimension of the state space) while requiring storage of only two state vectors, and a third auxiallary vector. Within the sampler one needs to operate by the precision matrix but there is no need to store the matrix or factorize it. Hence the congugate direction sampler is useful in high dimensional problems where forming the precision matrix is impractical or inconvenient. The real goal of this work is to provide an automatically efficient proposal distribution for the (t-walk version of the) Metropolis-Hastings MCMC algorithm. I will speculate on that possibility if I have not worked it out by the time of this talk.

Using decentralized reputation systems is a promising approach to ensuring cooperation and fairness in Mobile Ad-Hoc Networks. However, they are vulnerable to liars and robustness has not been analyzed in detail. We provide a first step to the analysis of a reputation system based on a deviation test. Nodes accept second hand information only if this does not differ too much from their reputation values. We show that the system exhibits a phase transition. In the subcritical regime it is robust and lying has not effect. In the supercritical regime lying does have an impact. We compute the critical values via a mean-field approach and use simulations to verify our results. We obtain conditions for the deviation test to make the reputation system robust and provide guidelines for a good choice of parameters.