Sigmoid AI (Page 3)

Balancing datasets using Crucio Safe-Level-SMOTE

Balancing datasets using Crucio Safe-Level-SMOTE

Unbalanced data In simple terms, an unbalanced dataset is where the target variable has more observations in one specific class than the others. An example can be a dataset for transactions: usually, 99% of this set consists of simple, not fraudulent transactions, and only 1% of them are fraud. What

Balancing datasets using Crucio MWMOTE

Balancing datasets using Crucio MWMOTE

Nowadays the majority of data sets in the industry are unbalanced. Meaning that a class has a higher frequency than others. Very often classifiers in such cases due to the unbalance of the data predict all samples as the most frequent class. To solve this problem we decided at Sigmoid

Using Crucio SMOTE and Clustered Undersampling Technique for unbalanced datasets

Using Crucio SMOTE and Clustered Undersampling Technique for unbalanced datasets

Unbalanced data In simple terms, an unbalanced dataset is one in which the target variable has more observations in one specific class than the others. An example can be a dataset for transactions: usually, 99% of this set consists of simple, not fraudulent transactions, and only 1% of them are

Generate new samples non-linearly using Crucio TKRKNN

Generate new samples non-linearly using Crucio TKRKNN

Nowadays the majority of data sets in the industry are unbalanced. Meaning that a class has a higher frequency than others. Very often classifiers in such cases due to the unbalance of the data predict all samples as the most frequent class. To solve this problem we decided at Sigmoid

Using Crucio SMOTENC for unbalanced datasets

Using Crucio SMOTENC for unbalanced datasets

Unbalanced data In simple terms, an unbalanced dataset is one in which the target variable has more observations in one specific class than the others. An example can be a dataset for transactions: usually, 99% of this set consists of simple, not fraudulent transactions, and only 1% of them are

Balancing data sets with Crucio ADASYN

Balancing data sets with Crucio ADASYN

Nowadays the majority of data sets in the industry are unbalanced. Meaning that a class has a higher frequency than others. Very often classifiers in such cases due to the unbalance of the data predict all samples as the most frequent class. To solve this problem we decided at Sigmoid

Model immunologic system with Crucio ICOTE

Model immunologic system with Crucio ICOTE

Nowadays the majority of data sets in the industry are unbalanced, meaning that a class has a higher frequency than others. Very often classifiers in such cases due to the unbalance of the data predict all samples as the most frequent class. To solve this problem we decided at Sigmoid

Use Entropy to find the where NaN
if informative with Kydavra
ShannonSelector

Use Entropy to find the where NaN if informative with Kydavra ShannonSelector

NaN values are one of the biggest problems in Machine Learning. However, problems are coming not from its presence, but from not knowing what they are meaning. Sometimes it is a full join that generates NaN values, in other cases, it means an imperfection of a sensor, in other cases,

Select features taking in account
distribution with Kydavra M3USelector

Select features taking in account distribution with Kydavra M3USelector

In the last article, we discussed the MUSESelector. This kydavra selector performs feature selection based on a data frame. The biggest drawback of this method is that it is good only for binary classification problems. There comes into play an extension of this method — M3U — (Minimum Mean Minimum Uncertainty), implemented

Select features taking in account
distribution with Kydavra MUSESelector

Select features taking in account distribution with Kydavra MUSESelector

One of the most intuitive ways to select features would be to find how much the distribution of the classes is different from each other. However on some intervals, the distribution of the feature by the classes can be different, but on other intervals, it can be practically the same.

Filter out the noise from your data with Kydavra PCAFilter

Filter out the noise from your data with Kydavra PCAFilter

Principal Component Analysis is known as one of the most popular dimension reduction techniques. However few know that it has a very interesting property — the reduced data can be brought back to the original dimension. Even more, the data brought back to its original size is more cleaned. So, at

Using Kydavra ElasticNetSelector for feature selection

Using Kydavra ElasticNetSelector for feature selection

Even if we don’t leave in the world of Harry Potter, now everyone can learn the unforgivable spells to use some not easy algorithms just by typing 3 lines of code. This can be done by using the Kydavra library for python. Using Kydavra ElasticNetSelector. To install Kydavra we

Reduce the dimensionality of your data with Kydavra LDAReducter

Reduce the dimensionality of your data with Kydavra LDAReducter

Many times, we have some features that are strongly correlated with the target column. However, sometimes they are correlated with each other, generating in such a way the problem of multicollinearity. One way is to reduce one of these columns. But, we at sigmoid want to propose to you a

Using Kydavra MixerSelector for feature selection

Using Kydavra MixerSelector for feature selection

Usually, when we try to select the best features of the dataset, we choose different spells(algorithms) to see a difference, and select the best one, so why don’t we try to mix them all up and combine the power them all in one spell. This is where the

Using Kydavra ANOVASelector for feature selection

Using Kydavra ANOVASelector for feature selection

Very often we can look at classification as a problem of finding differences between 2 groups. Before Machine Learning Statisticians were doing it quite a lot. Mostly they used such metrics as mean, variation, and standard deviation. However, it was a time-consuming process, and couldn’t be done with many

Find features that really explains your data with Kydavra PValueSelector

Find features that really explains your data with Kydavra PValueSelector

Features selection is a very important part of machine learning development, It allows you to keep your models as simple as possible keeping at the same time, as much information as possible. Unfortunately, sometimes it can require high mathematical knowledge and good practical skills in programming. However, at Sigmoid we

Regression Feature Selection using the Kydavra LassoSelector

Regression Feature Selection using the Kydavra LassoSelector

We all know the Occam’s Razor: From a set of solutions took the one that is the simplest. This principle is applied in the regularization of the linear models in Machine Learning. L1-regularisation (also known as LASSO) tend to shrink wights of the linear model to 0 while L2-regularisation

Solving categorical feature selection with Kydavra ChiSquaredSelector

Solving categorical feature selection with Kydavra ChiSquaredSelector

So how we said in previous articles about Kydavra library, Feature selection is a very important part of Machine Learning model development. Unfortunately, there is not only one unique way to get the ideal model, mostly because of the fact that data almost every time has different forms, but this

Solve binary classification with PointBiserialCorrSelector

Solve binary classification with PointBiserialCorrSelector

The majority of machine learning problems in today’s world are the classification ones. Mostly data scientists and machine learning engineers use Correlations as Pearson or Spearman to find features that correlate the most with the predicted value. However, these types of correlations work better on continuous-continuous pair of features.

Gaussian Mixture Models:
implemented from scratch

Gaussian Mixture Models: implemented from scratch

From the rising of the Machine Learning and Artificial Intelligence fields Probability Theory was a powerful tool, that allowed us to handle uncertainty in a lot of applications, from classification to forecasting tasks. Today I would like to talk with you more about the use of Probability and Gaussian distribution

Easy to use Correlation Feature Selection with Kydavra

Easy to use Correlation Feature Selection with Kydavra

Almost every person in data science or Machine Learning knows that one of the easiest ways to find relevant features for predicted value y is to find the features that are most correlated with y. However few (if not a mathematician) know that there are many types of correlation. In

Applying Darwinian Evolution to feature selection with Kydavra GeneticAlgorithmSelector

Applying Darwinian Evolution to feature selection with Kydavra GeneticAlgorithmSelector

The development of machine learning implies a lot of maths. But sometimes during feature selection phase maths sometimes can’t give an exact answer (because of the structure of data, it’s source, and many other causes). Then in-game enter the Programming tricks, mostly brute force methods :). Genetic algorithms are

The most important part of Artificial Intelligence Systems development

The most important part of Artificial Intelligence Systems development

Nowadays, AI-related fields like Data Science and Machine Learning have become mainstream. Practically everyone can now become a data scientist. That’s, of course, a good thing, but unfortunately, people forget about what an AI system is. I prefer the notion by Patrick Winston from MIT: Artificial Intelligence (or Intelligent

Welcome to our community!

Welcome to our community!

We are Sigmoid – a non-governmental, research and development (R&D) organization, which aims to develop the field of Machine Learning and Artificial Intelligence in Moldova, support and train personnel in the respective fields, as well as offer the experience of working in an organization. Our main goals are: 1.