AI/ML

Posts related to Machine Learning, Deep Learning and Data Science.

AI/ML

Feature Selection using the Kydavra FisherSelector

A Brief Review of Fisher Score: Fisher Score is one of the most widely used supervised feature selection methods. However, it selects each feature independently according to their scores under the Fisher Criterion, which leads to a suboptimal subset of features. Kydavra 0.3's latest release provides a

AI/ML

Relief-based feature selection using ReliefFSelector

Selecting the right set of features to be used for data modelling has been shown to improve the performance of supervised and unsupervised learning, to reduce computational costs such as training time or required resources, in the case of high-dimensional input data to mitigate the curse of dimensionality. Computing and

AI/ML

Placing Models on Streamlit

What is streamlit? Streamlit is a web application framework that helps you build and develop Python-based web applications that can be used to share analytics results, build complex interactive experiences, and illustrate new machine learning models. On top of that, developing and deploying Streamlit apps is incredibly fast and flexible,

AI/ML

Bring your data to normal distribution with imperio YeoJohnsonTransformer

One of the biggest problems with data in Data Science is its distribution, it almost every single time isn’t normal. It happens because we cannot have all samples in the world in one data set. However, there exists a bunch of methods that can change that. In one of

AI/ML

Whiten your data with imperio ZCATransformer

Feature engineering is the process of transforming your input data in such a way that it will be more representative of the Machine Learning Algorithms. However, it is very often forgotten because of the inexistence of an easy-to-use package. That’s why we decided to create the one — imperio, the

AI/ML

Retain Target Information with imperio TargetImputationTransformer

As we said in the article about FrequencyImputationTransoferm, usually, categorical values are replaced by integer numbers. However, this approach is very dangerous for linear models, because of the false correlation that may appear. A step forward from this technique is One Hot Encoding (or Dummy variables). But even Dummy variables

AI/ML

Reduce the outliers influence with imperio SpatialSignTransformer

Outliers are a big problem of almost all types of Machine Learning algorithms. Would be fine if somehow with a data preprocessing technique would be possible to eliminate or at least to reduce their influence on data. You can consider yourself lucky because it exists — Spatial Sign is its name.

AI/ML

Whiten your data with imperio WhiteningTransformer

Feature engineering is the process of transforming your input data in such a way that it will be more representative of the Machine Learning Algorithms. However, it is very often forgotten because of the inexistence of an easy-to-use package. That’s why we decided to create the one — imperio, the

AI/ML

Transform your categorical columns with imperio SmoothingTransformer

Feature engineering is the process of transforming your input data in such a way that it will be more representative of the Machine Learning Algorithms. However, it is very often forgotten because of the inexistence of an easy-to-use package. That’s why we decided to create the one — imperio, the

AI/ML

Reduce data diversity with imperio LogTransformer

One of the biggest problems with data in Data Science is its distribution, it almost every single time isn’t normal. It happens because we cannot have all samples in the world in one data set. However, there exists a bunch of methods that can change that. Today we will

AI/ML

Present the categories importance with imperio FrequencyImputationTransformer

Usually, categorical values are replaced by integer numbers. However, this approach is very dangerous for linear models, because of the false correlation that may appear. A step forward from this technique is One Hot Encoding (or Dummy variables). But even Dummy variables have its drawbacks — the matrix is becoming very

AI/ML

Combine transformers with imperio CombinatorTransformer

Very often we would like to combine different transformations on different columns of our feature matrix in a pipeline. That need motivated us to a special module that gives you this opportunity in imperio — CombinatorTransformer. How CombinatorTransformer works? CombinatorTransformer allows you to apply a specific feature transformation on a specific

AI/ML

Normalize your data distribution with imperio BoxCoxTransformer

Feature engineering is the process of transforming your input data in such a way that it will be more representative of the Machine Learning Algorithms. However, it is very often forgotten because of the inexistence of an easy-to-use package. That’s why we decided to create the one — imperio, the

AI/ML

Using Crucio SMOTETOMEK for balancing data

This article will be about a combination of the two most powerful algorithms used for oversampling and undersampling imbalanced datasets, SMOTE and Tomek Links. I talked about SMOTE in my previous article, so you should check it out because I will now rely more on the Tomek Links algorithm. So

AI/ML

Using Crucio SMOTEENN for balancing data

This article will be about a combination of the two most powerful algorithms used for oversampling and undersampling imbalanced datasets, SMOTE and ENN. I talked about SMOTE in one of my previous articles, so you should check it out because I will now rely more on the ENN algorithm. How

AI/ML

Using Crucio SMOTE for balancing data

This will be a new series of articles related to our brand new Python library made with love by Sigmoid. Its name is just like the second unforgivable spell from the Harry Potter series, Crucio. This library was created specifically for unbalanced data sets, with a lot of different methods

AI/ML

Generate synthetic samples using Crucio MTDF

Nowadays the majority of data sets in the industry are unbalanced. Meaning that a class has a higher frequency than others. Very often classifiers in such cases due to the unbalance of the data predict all samples as the most frequent class. To solve this problem we decided at Sigmoid

AI/ML

Balancing datasets using Crucio Safe-Level-SMOTE

Unbalanced data In simple terms, an unbalanced dataset is where the target variable has more observations in one specific class than the others. An example can be a dataset for transactions: usually, 99% of this set consists of simple, not fraudulent transactions, and only 1% of them are fraud. What

AI/ML

Balancing datasets using Crucio MWMOTE

Nowadays the majority of data sets in the industry are unbalanced. Meaning that a class has a higher frequency than others. Very often classifiers in such cases due to the unbalance of the data predict all samples as the most frequent class. To solve this problem we decided at Sigmoid

AI/ML

Using Crucio SMOTE and Clustered Undersampling Technique for unbalanced datasets

Unbalanced data In simple terms, an unbalanced dataset is one in which the target variable has more observations in one specific class than the others. An example can be a dataset for transactions: usually, 99% of this set consists of simple, not fraudulent transactions, and only 1% of them are

AI/ML

Generate new samples non-linearly using Crucio TKRKNN

Nowadays the majority of data sets in the industry are unbalanced. Meaning that a class has a higher frequency than others. Very often classifiers in such cases due to the unbalance of the data predict all samples as the most frequent class. To solve this problem we decided at Sigmoid

AI/ML

Using Crucio SMOTENC for unbalanced datasets

Unbalanced data In simple terms, an unbalanced dataset is one in which the target variable has more observations in one specific class than the others. An example can be a dataset for transactions: usually, 99% of this set consists of simple, not fraudulent transactions, and only 1% of them are

AI/ML

Balancing data sets with Crucio ADASYN

Nowadays the majority of data sets in the industry are unbalanced. Meaning that a class has a higher frequency than others. Very often classifiers in such cases due to the unbalance of the data predict all samples as the most frequent class. To solve this problem we decided at Sigmoid

AI/ML

Model immunologic system with Crucio ICOTE

Nowadays the majority of data sets in the industry are unbalanced, meaning that a class has a higher frequency than others. Very often classifiers in such cases due to the unbalance of the data predict all samples as the most frequent class. To solve this problem we decided at Sigmoid

AI/ML

Select features taking in account distribution with Kydavra M3USelector

In the last article, we discussed the MUSESelector. This kydavra selector performs feature selection based on a data frame. The biggest drawback of this method is that it is good only for binary classification problems. There comes into play an extension of this method — M3U — (Minimum Mean Minimum Uncertainty), implemented