Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bug- spitting in feature projection is splitting in time #128

Open
WeissShahaf opened this issue Aug 26, 2019 · 9 comments
Open

bug- spitting in feature projection is splitting in time #128

WeissShahaf opened this issue Aug 26, 2019 · 9 comments

Comments

@WeissShahaf
Copy link

when splitting clusters using feature projection, the split always seems to be a time cut off
in addition to that, the clusters which appear to be distinct, are still present after cutting.

this happens on various computers, and recording sessions.

GL383_20190815_OF_g0_t0.imec0.ap.prm.zip

If you have a usage or support question, come join our Gitter and ask your question there.
If you found a bug, please provide all of the following:

Output of ver

ver

MATLAB Version: 9.6.0.1072779 (R2019a)
MATLAB License Number: 40523909
Operating System: Microsoft Windows 10 Enterprise Version 10.0 (Build 18362)
Java Version: Java 1.8.0_181-b13 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode

MATLAB Version 9.6 (R2019a)
Simulink Version 9.3 (R2019a)
5G Toolbox Version 1.1 (R2019a)
AUTOSAR Blockset Version 2.0 (R2019a)
Aerospace Blockset Version 4.1 (R2019a)
Aerospace Toolbox Version 3.1 (R2019a)
Antenna Toolbox Version 4.0 (R2019a)
Audio Toolbox Version 2.0 (R2019a)
Automated Driving Toolbox Version 2.0 (R2019a)
Bioinformatics Toolbox Version 4.12 (R2019a)
Communications Toolbox Version 7.1 (R2019a)
Computer Vision Toolbox Version 9.0 (R2019a)
Control System Toolbox Version 10.6 (R2019a)
Curve Fitting Toolbox Version 3.5.9 (R2019a)
DSP System Toolbox Version 9.8 (R2019a)
Data Acquisition Toolbox Version 4.0 (R2019a)
Database Toolbox Version 9.1 (R2019a)
Datafeed Toolbox Version 5.8.1 (R2019a)
Deep Learning Toolbox Version 12.1 (R2019a)
Econometrics Toolbox Version 5.2 (R2019a)
Embedded Coder Version 7.2 (R2019a)
Filter Design HDL Coder Version 3.1.5 (R2019a)
Financial Instruments Toolbox Version 2.9 (R2019a)
Financial Toolbox Version 5.13 (R2019a)
Fixed-Point Designer Version 6.3 (R2019a)
Fuzzy Logic Toolbox Version 2.5 (R2019a)
GPU Coder Version 1.3 (R2019a)
GUI Layout Toolbox Version 2.3.4 (R2018b)
Global Optimization Toolbox Version 4.1 (R2019a)
HDL Coder Version 3.14 (R2019a)
HDL Verifier Version 5.6 (R2019a)
Image Acquisition Toolbox Version 6.0 (R2019a)
Image Processing Toolbox Version 10.4 (R2019a)
Instrument Control Toolbox Version 4.0 (R2019a)
LTE HDL Toolbox Version 1.3 (R2019a)
LTE Toolbox Version 3.1 (R2019a)
MATLAB Coder Version 4.2 (R2019a)
MATLAB Compiler Version 7.0.1 (R2019a)
MATLAB Compiler SDK Version 6.6.1 (R2019a)
MATLAB Report Generator Version 5.6 (R2019a)
Mapping Toolbox Version 4.8 (R2019a)
Mixed-Signal Blockset Version 1.0 (R2019a)
Model Predictive Control Toolbox Version 6.3 (R2019a)
Model-Based Calibration Toolbox Version 5.6 (R2019a)
OPC Toolbox Version 4.0.7 (R2019a)
Optimization Toolbox Version 8.3 (R2019a)
Parallel Computing Toolbox Version 7.0 (R2019a)
Partial Differential Equation Toolbox Version 3.2 (R2019a)
Phased Array System Toolbox Version 4.1 (R2019a)
Powertrain Blockset Version 1.5 (R2019a)
Predictive Maintenance Toolbox Version 2.0 (R2019a)
RF Blockset Version 7.2 (R2019a)
RF Toolbox Version 3.6 (R2019a)
Reinforcement Learning Toolbox Version 1.0 (R2019a)
Risk Management Toolbox Version 1.5 (R2019a)
Robotics System Toolbox Version 2.2 (R2019a)
Robust Control Toolbox Version 6.6 (R2019a)
Sensor Fusion and Tracking Toolbox Version 1.1 (R2019a)
SerDes Toolbox Version 1.0 (R2019a)
Signal Processing Toolbox Version 8.2 (R2019a)
SimBiology Version 5.8.2 (R2019a)
SimEvents Version 5.6 (R2019a)
Simscape Version 4.6 (R2019a)
Simscape Driveline Version 2.16 (R2019a)
Simscape Electrical Version 7.1 (R2019a)
Simscape Fluids Version 2.6 (R2019a)
Simscape Multibody Version 6.1 (R2019a)
Simulink 3D Animation Version 8.2 (R2019a)
Simulink Check Version 4.3 (R2019a)
Simulink Code Inspector Version 3.4 (R2019a)
Simulink Coder Version 9.1 (R2019a)
Simulink Control Design Version 5.3 (R2019a)
Simulink Coverage Version 4.3 (R2019a)
Simulink Design Optimization Version 3.6 (R2019a)
Simulink Design Verifier Version 4.1 (R2019a)
Simulink Desktop Real-Time Version 5.8 (R2019a)
Simulink PLC Coder Version 3.0 (R2019a)
Simulink Real-Time Version 6.10 (R2019a)
Simulink Report Generator Version 5.6 (R2019a)
Simulink Requirements Version 1.3 (R2019a)
Simulink Test Version 3.0 (R2019a)
SoC Blockset Version 1.0 (R2019a)
Spreadsheet Link Version 3.4.1 (R2019a)
Stateflow Version 10.0 (R2019a)
Statistics and Machine Learning Toolbox Version 11.5 (R2019a)
Symbolic Math Toolbox Version 8.3 (R2019a)
System Composer Version 1.0 (R2019a)
System Identification Toolbox Version 9.10 (R2019a)
Text Analytics Toolbox Version 1.3 (R2019a)
Trading Toolbox Version 3.5.1 (R2019a)
Vehicle Dynamics Blockset Version 1.2 (R2019a)
Vehicle Network Toolbox Version 4.2 (R2019a)
Vision HDL Toolbox Version 1.8 (R2019a)
WLAN Toolbox Version 2.1 (R2019a)
Wavelet Toolbox Version 5.2 (R2019a)

Output of gpuDevice

gpuDevice(1)
ans =
CUDADevice with properties:

                  Name: 'Quadro RTX 8000'
                 Index: 1
     ComputeCapability: '7.5'
        SupportsDouble: 1
         DriverVersion: 10.1000
        ToolkitVersion: 10
    MaxThreadsPerBlock: 1024
      MaxShmemPerBlock: 49152
    MaxThreadBlockSize: [1024 1024 64]
           MaxGridSize: [2.1475e+09 65535 65535]
             SIMDWidth: 32
           TotalMemory: 4.8318e+10
       AvailableMemory: 4.0199e+10
   MultiprocessorCount: 72
          ClockRateKHz: 1770000
           ComputeMode: 'Default'
  GPUOverlapsTransfers: 1
KernelExecutionTimeout: 0
      CanMapHostMemory: 1
       DeviceSupported: 1
        DeviceSelected: 1

The command you used

(e.g., jrc detect-sort myparam.prm)
jrc manual

If manually sorting, what it is you tried to do

split using Feature projection

Console output

2019-08-26 09:00:17 Computing cluster mean waveforms...
2019-08-26 09:00:17 Finished computing cluster mean waveforms (took 0.13 s)
2019-08-26 09:00:17 Computing waveform correlation...
2019-08-26 09:00:17 Finished computing waveform correlation (took 0.12 s)
2019-08-26 09:00:17 Computing cluster quality scores...
2019-08-26 09:00:17 Finished computing cluster quality scores (took 0.02 s)
@aliddell
Copy link
Collaborator

Hi @WeissShahaf, can you let me know what's the git commit hash of the version you're running, and maybe show me a screenshot?

@WeissShahaf
Copy link
Author

i'm running the latest commit, SHA 5e8f020

before
before1
before2
before3
before4
before5
after1
after2
after3
after3a
after5

@aliddell
Copy link
Collaborator

Makes sense now. This could be a bug in how the features for the feature projection view are sampled. Let me investigate and get back to you.

@aliddell
Copy link
Collaborator

Hey, please give the latest commit a pull and let me know if you still see this problem.,

@WeissShahaf
Copy link
Author

-some weird behavior now: projection plot now is displaying spikes for some clusters but not others.
though it shows background spikes for all:

1
2

-when i try splitting it fails:

Warning: Failed to split: Operands to the || and && operators must be convertible to logical scalar values.

In jrclust.curate.CurateController/splitCluster (line 16)
In jrclust.curate.CurateController/splitPoly (line 68)
In jrclust.curate.CurateController/keyPressFigProj (line 65)
In jrclust.curate.CurateController/plotAllFigures>@(varargin)obj.keyPressFigProj(varargin{:}) (line 23)

-Also display wave waveforms is weird- mean waveforms seem flat, when waveforms are clearly not:

U235_waveforms
u2_mean
u2_waveforms
U235_mean

and when i try to split it fails

@aliddell
Copy link
Collaborator

Can you pull the latest and run the following script for me?

hCfg = jrclust.Config('/path/to/your/params.prm'); % load your prm file
hCfg.autoRecover = 1; % set an optional parameter to try automatically fixing this
hJRC = JRC(hCfg);
hJRC.loadFiles(); % load bin files, res.mat, try to fix any data inconsistencies
hJRC.curate(); % inspect the results

@WeissShahaf
Copy link
Author

it does split after the patch, sometimes, but still splitting in time.
other cluster still fail to split.

console output:
Warning: Failed to split: split indices exceed bounds

In jrclust.curate.CurateController/splitCluster (line 21)
In jrclust.curate.CurateController/splitPoly (line 95)
In jrclust.curate.CurateController/keyPressFigProj (line 65)
In jrclust.curate.CurateController/plotAllFigures>@(varargin)obj.keyPressFigProj(varargin{:}) (line 23)

for some recordings the patch fails.
especially if i try it after starting manual curation. then i get an error message about unstable statistics and that curation will likely not be possible.

@aliddell
Copy link
Collaborator

aliddell commented Sep 3, 2019

That surprises me. I'm inclined to think it's something particular to your dataset. May I take a look at it?

Also, you mentioned it only sometimes splits? What happens when it doesn't?

@WeissShahaf
Copy link
Author

if the split fails it says operation failed.

Warning: Failed to split: split indices exceed bounds

In jrclust.curate.CurateController/splitCluster (line 21)
In jrclust.curate.CurateController/splitPoly (line 95)
In jrclust.curate.CurateController/keyPressFigProj (line 65)
In jrclust.curate.CurateController/plotAllFigures>@(varargin)obj.keyPressFigProj(varargin{:}) (line 23)

here is a link to an example dataset test folder:
https://www.dropbox.com/sh/36jcwu2n5qkkz5l/AADFZmVmHGaI40jBOXyUOHDWa?dl=0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@aliddell @WeissShahaf