shrishtishettymaths

The ARIMA (Autoregressive Integrated Moving Average) model is a popular statistical method for time series forecasting that captures the dynamics of the series through three main parameters: AR (p), I (d), and MA (q).
AR (Autoregression): Refers to the use of previous values in the time series to predict future values.
I (Integrated): Represents the differencing of raw observations to make the time series stationary, which means that the series has constant mean and variance over time.
MA (Moving Average): Incorporates the dependency between an observation and a residual error from a moving average model applied to lagged observations.
The lines in each graph represent:
Blue Line (Actual): The actual observed values from the dataset.
Orange Line (Fitted): The values predicted by the ARIMA model.

The Partial Autocorrelation Function (PACF) plots show the partial correlation of each time series with its own lagged values, controlling for the values of the time series at all shorter lags. This is helpful in identifying the order of the autoregressive (AR) part of an ARIMA model. Here’s how to interpret these plots:

Autocorrelation Function (ACF) plots for various time series data. These plots show the correlation of the time series with its own past values at different lags (time intervals). The blue shaded area in the plot represents the confidence interval, typically set at 95%. Correlation values outside of this area are considered statistically significant.

Time series plots of differenced economic indicators, each with a p-value indicating the significance of a statistical test, likely a unit root test such as the Augmented Dickey-Fuller (ADF) test. Here’s an analysis of what each plot represents:

Augmented Dickey-Fuller (ADF) tests the null hypothesis that a unit root is present in a time series. Each plot is labeled with an “ADF Statistic” and a “p-value,” which are used to determine whether a time series is stationary.

The joint kernel density estimate (KDE) plots illustrate the relationship between different economic indicators and the total number of jobs, using data from the provided dataset. Here’s an analysis of each plot:

Understanding the general pattern in order to determine Boston’s tourism peak times:

The monthly passenger data at Logan Airport paints a vivid picture of tourism trends in Boston. Throughout the year, we can see distinct peaks and valleys in the number of passengers, which correspond to the highs and lows of the tourism season. This cyclical pattern is a key indicator of when the city experiences its highest influx of visitors.

Moreover, by looking at the graph year-over-year, we can observe whether tourism in Boston is growing, remaining stable, or facing a decline. Such insights are crucial for stakeholders in the tourism industry, as they provide a clear view of the most popular times for tourists in the city. This information can be invaluable for planning purposes, whether it’s for staffing needs, marketing campaigns, or resource management. In essence, the passenger numbers at Logan Airport offer a reliable barometer for understanding and anticipating the dynamics of tourism in Boston.

I began examining information from Analyze Boston, the open data repository for the City of Boston. This is a remnant dataset of economic indicators from the Boston Planning and Development Authority (BPDA), which was in charge of organizing and directing inclusive growth in the City of Boston, that were recorded monthly between January 2013 and December 2019. A wide range of economic data on employment, housing, travel, and real estate development are gathered and analyzed by BPDA. I was able to correctly eliminate all null value.

The decision tree divides the data into branches that helped to reveal patterns and relationships. It can show whether individuals of a particular race in a particular age group are more or less likely to engage in police shootings. The decision tree lets us see how different types of shootings by police have occurred. The results of using a decision tree provides a clear and descriptive way to understand the complex relationships in our given data. It will also help us identify specific situations where police shootings are more common in certain age groups and ethnic groups. This can be a powerful tool for uncovering biases or trends that may not be immediately obvious by simply looking at raw data.

DBSCAN (Density-Based Spatial Clustering of Applications with Noise): This algorithm forms clusters based on the density of data points. It groups points that are closely packed together and marks points in low-density regions as outliers. DBSCAN can find clusters of any shape and is good at separating high-density clusters from noise in the data.

K-means: This method groups data into ‘k’ number of clusters by reducing the distance between data points and the center of their assigned cluster. The ‘means’ in the name refers to averaging the data points to find the center of the cluster. It’s good for spherical-shaped clusters.

K-medoids: Similar to K-means, but instead of using the mean, it uses actual data points as the center of the cluster, known as medoids. This method is more robust to noise and outliers compared to K-means because medoids are less influenced by extreme values.

Effect Size: Using effect size is similar to measuring the robustness or plausibility of an observed pattern. It tells us not only that there is a difference, but how big that difference is. Using effect sizes for age differences between racial groups in police shootings will not only tell us that there are significant differences, but also allow us to understand how large those differences are in terms of benefits. This helps us understand the true significance of the findings, rather than simply recognizing that the differences are not due to chance.

Statistical Significance: It will helps us determine whether findings from data, as the ages of different groups of people killed by police, may be accurate or could have just happened. If something is statistically significant it means that we are pretty sure that the patterns we see in the data (such as one group being younger or older than another) are real, not just a random so if we use statistical significance then age differences between groups are real.

If we want to see how age and race might affect certain outcomes, such as police contacts, it is inefficient to do a bunch of subtests for each category.  ANOVA is a technique that allows us to look at all the puzzles at once. It is a single, simplified test that can tell us whether age, race, or a mixture of both has an effect giving us a clear answer.

Variance tells us how spread out the ages are in each racial group of people killed by police. If everyone were the same age, variance would be zero because there’s no spread. But in real life, ages vary, so we get a number that tells us how much they differ from the average age. For different racial groups have different variances in ages. This is important because when we use methods like ANOVA (Analysis of Variance), we assume that these variances are roughly the same across groups. But that’s not the case here, which means we might need to use different statistical methods that don’t make this assumption to accurately understand the data.

ANOVA and t-tests are used to obtain p-values, which are then used to calculate the chance of observing a significant mean difference. From a Bayesian standpoint, this is so-called frequentist method of determining a large difference in means is considered fundamentally incorrect, particularly in some cases it is evident that the null hypothesis is false. However, a Bayesian approach to inference for the age and race data appears to be doomed mainly because, although the data exist, we do not know of any previous distributions, have any idea how those prior distributions might look, or have any idea in what sense the age and race data is evidence of change.

The average age of Hispanics group is 32 years old, meaning that half of the population is younger and the other half older. The mean age of 33.59 years is more over the average, suggesting some skewness in the age distributions. The average age of this group is indicated by the standard deviation, which is 10.74 years. The distribution’s skewness, which measures 0.803, indicates a moderate right skew with a greater proportion of younger than elderly people. The age distribution for Hispanics is significantly higher and has more tail, as indicated by the kurtosis value of 3.725, which is higher than the kurtosis of a normal distribution.

The Native American group shows an even age distribution with a mean age of 32 years. This group has a slightly larger age group than the Hispanic group, with a median age of 8.949 years. Even though it is less than the Hispanic distribution, a skewness value of 0.565 indicates a minor skewness to the right. Since the age distribution has a somewhat smaller tail and a less prominent peak than the normal distribution, the kurtosis value of 2.883 suggests that age is undervalued.

Importing data for age and race only for those which has values ​​for both variables. The Asian population median age according to age data is 35 years old. With a standard deviation of 11.5921, the ages range from an average of about 35.96 years. This dispersion shows that the age statistics vary by 134.377. For Asian population the skewness, which measures the asymmetry of the age distribution is computed as 0.327765. The kurtosis gives information on the age distribution’s tails and sharpness which is 2.35263. The median age of Black population according to age data is 31 years old. With 32.9281 years on average, the average age is slightly more. With a variance of 129.701, the standard deviation shows the distribution of ages is 11.3886. With a skewness of 0.962894, the age distribution of the Black population is more skewed which indicates asymmetry. Compared to the Asian population, the kurtosis of 3.81164 indicates a distinct peak sharpness and tail.

The age of individuals varies from 13 to 88 years. The average age is 31.7 years, slightly below the mean age of 32.7 years. The standard deviation representing the distance around the mean is about 11.4 years. Since the skewness value is about 0.99, the data appears to be slightly skewed to the right. A kurtosis value of 3.91 is usually used to indicate a relatively high distribution, which deviates from the kurtosis of 3 for a normal distribution. This could indicate a discrepancy or a certain age.

Looking at the statistics of the age distribution after removing the ages from the cells that do include age values. Because the mean and median are different, the distribution of ages is significantly skewed to the right; in fact the skewness is about 0.73. The kurtosis is near to 3, indicating that there are no significant fat tails or peaks around the mean. After checking the percentage of the right tail of the age distribution lies more than 2 standard deviations from the mean which is somewhat greater than what get for standard normal distribution.

Several separate tests comparing groups based on factors such as age and race can be repetitive and time-consuming. Instead, the ANOVA test is a much better method. This test allows us to compare all groups at once, making analysis easier and faster, and helps us to understand whether and how these factors are related.

The data presented in the residual models of diabetes and inactivity provide valuable insight into the residual characteristics in the context of a background sample. By using these data we can collectively provide an assessment of the  linear regression model. The residuals are shown to have a moderate spread, exhibit symmetry at the center, and have few outliers. While these characteristics are important for understanding model performance, they do not directly reflect the strength or ambiguity of the relationship between diabetes and inactivity, which would require examination of regression coefficients and R-squared values, if there is any.

The bootstrap is a versatile and effective statistical tool that may be used to calculate the level of uncertainty surrounding a certain estimate or statistical learning technique. It can offer a confidence range for a coefficient or an estimate of the standard error of that coefficient. The bootstrap method allows the computer to recreate the process of getting fresh data sets, allowing us to estimate our estimate’s variability without having to create more samples.

Bootstrapping creates many different versions of the dataset, which allows us to see how different subsets of data can affect the result. Bootstrapping also helps to get a more accurate understanding of the overall given diabetes population and it can also help to calculate more accurate confidence intervals. Bootstrapping will helps us to assess the model performs on new data that its not seen

The t-test is a parametric test of difference, meaning that it makes the same assumptions about your data as other parametric tests. The t test assumes your  data is independent, is normally distributed and have a similar amount of variance within each group being compared. A t-test helps determine whether the difference in mean between two groups is due to real effect or if it has occurred randomly.

The Monte Carlo permutation test is a technique used to determine whether the observed difference between two groupsis real or if it has occurred by chance. It is very helpful while dealing with complex data.

When we carry out 5-fold cross-validation on the given data which consists data of obesity, inactivity and diabetes.There are a 354 data points of all 3 variables. With increasing model complexity, the training error often decreases and tends to underestimate the test error. Randomly divide the data into 5 roughly equal groups of 71, 71, 71, and 70, which we then use for cross-validations to determine the test error.  Then dividing the labelled data in which no repeats are present, into 5 equal groups  of 71, 71, 71,71 and 70. Then using 5 training data and test data sets to build, polynomial models on the training data and find the mean square error on the test data.

K-fold Cross-Validation is a widely used approach for estimating test error. The idea is to randomly divide the data into K equal-sized parts. We leave out part k, fit the model to the other K-1 parts, and then obtain predictions for the left-out kth part. This is done in turn for each part k = 1,2…..k, and then the results are combined. We cannot apply cross-validation in step 2 directly without applying it in step 1. Because it would ignore the fact that in step 1, the procedure has already seen the labels of the training data, and made use of them. This is a form of training and must be included in the validation process.

In the given CDC 2018 diabetes data, we discovered that how your weight (BMI) affects your chances of getting diabetes depends on your age. It’s kind of like BMI has a different role for younger and older people. Instead of using straight lines, we might have to use curvy lines when looking at the data. We use special math to handle these curvy lines. When the relationship between factors like your weight and diabetes isn’t a straight line, we need to use math that fits the curves to make it simpler to understand. We use step functions to catch these quick jumps, like when a signifiant increase in exercise suddenly has a big impact on risk.

Variance means the measurement of how spread out or dispersed the values in a dataset are. A smaller variance indicates that the data points are closer to the mean and less dispersed, while a larger variance suggests that the data points are more spread out from the mean. A t-test is a way to figure out if two sets of data are different from each other in a meaningful way, taking into account the variability within each group and the size of the samples. P-value shows if the data is normal. If it’s below 0.05, data might not be normal. If its above 0.05 then its close to normal. A pre-molt histogram shows the distribution of data before an event. A post-molt histogram shows data after the event to check for changes. A Monte Carlo permutation test that involves repeatedly shuffling the data to simulate random chance outcomes.

Linear regression model with more than one predictor variable is known as multiple linear regression. For Galton, “regression” referred only to the tendency of extreme data values to “revert” to the overall mean value. Correlation between predictor variables which is also known as factors. %inactivity and %obesity are considered as factors (predictor variable) for %diabetes. A high R-squared value determines how well the model fits in the observed data. Cross Validation is a technique to check error in data and to check overfitting.

The linear model’s significance in data science projects must be emphasized. By simplifying calculations, we can understand why we measure distance parallel to the Y-axis in linear models. It is advised to use transformations like log or exponential in linear models.

Heteroscedasticity is important to ensure that the assumptions of the model are met and that the results are valid. Breusch-Pagan test is used to test heteroscedasticity analytically. The Breusch–Pagan test, developed in 1979 by Trevor Breusch and Adrian Pagan. There are a couple of variants of the Breusch-Pagan test. It uses the residuals from the original linear regression of %diabetes against %inactivity. Calculate Pearson’s R-Squared  and then the calculate the p-value. A p-value is a peculiar value which is a method to assess whether the results of an experiment or study are statistically significant or whether they could have occurred by chance. We need to see the p-value is as large as n  where n = 1370 which is the number of %inactivity data points for a chi-squared distribution with degree of freedom. Typically, p-value is very small such as 0.05, it means it has strong evidence against the null hypothesis. In these types of cases, they choose to reject the null hypothesis that the original linear model is homoscedastic.

The Federal Information Processing Standards (FIPS ) common to all data sets shows 354 rows of data that contains information on all 3 variables %obesity, %inactivity and %diabetes. There are relatively large number of data points for both diabetes and inactivity. For the given %diabetes data has a kurtosis of approximately 4, which is slightly higher than the value of 3 for a normal distribution, and a quantile-quantile plot reveals a significant departure from normality and for the given %inactivity data has a kurtosis of about 2, which is somewhat lower the value of 3 for normal distribution. Kurtosis is critical. The linear least squares model is a technique used to identify the line that best illustrates the relationship between the variables, by minimizing the sum of the squared differences between the anticipated and observed values of the data points. In any linear model it is very important to examine residuals. Residuals represents the error or unexplained variations in the data. When the residual is examined it is noticed that the higher than 3 value of 4.07 for the kurtosis, and the quantile plot,  its indicates a deviation from normality for the residuals. This can create issue in testing for heteroscedasticity. Heteroscedasticity is important to ensure that the assumptions of the model are met and that the results are valid. Breusch-Pagan test is used to test heteroscedasticity analytically. When we plot residuals versus the predicted values from linear model, the fanning out of residuals as the fitted value gets large is an indicator that the linear model is not reliable hence it is heteroscedastic.

Welcome to UMassD WordPress. This is your first post. Edit or delete it, then start blogging!